The foundation for best practices in machine learning

ABN AMRO

Although the organisational promise and value of Machine Learning is great, a myriad of pitfalls currently accompany this modern practice of data science. The acknowledged operational, ethical, legal and governance risks have generated a need for a clear and thoughtful repository of best practices on how to responsibly govern, manage and implement Machine Learning (“responsible ML”). Ancillary commercial benefits of responsible ML are robust models and greater R&D productionalisation rates/success.

The Foundation for Best Practices in Machine Learning (non-profit) seeks to promote responsible ML through creating an open-sourced, freely accessible repository of best practices and associated guides. Its model and organisational guides look at both the technical and institutional requirements needed to promote responsible ML. Both blueprints touch on subjects such as “Fairness & Non-Discrimination”, “Representativeness & Specification”, “Product Traceability”, “Explainability” amongst other topics. Where the organisational guide relates to organisation-wide process and responsibilities (f.e. the necessity of setting proper product definitions and risk portfolios); the model guide details issues ranging from cost function specification & optimisation to selection function characterization, from disparate impact metrics to local explanations and counterfactuals. It also addresses issues concerning thorough product management.

These guidelines have been developed principally by senior ML engineers, data scientists, data science managers, and legal professionals for ML engineers, data scientists, data science managers, compliance professionals, legal practitioners, and, more broadly, management. The Foundation’s philosophy is that (a) context is key, and (b) responsible ML starts with prudent MLOps and product management.

Organizing data science