ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles

By Inioluwa Deborah Raji, Jingyi Yang. 2019

42 citations recorded by the source corpus.

We present the "Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles" (ABOUT ML) project as an initiative to operationalize ML transparency and work towards a standard ML documentation practice. We make the case for the project's relevance and effectiveness in consolidating disparate efforts across a variety of stakeholders, as well as bringing in the perspectives of currently missing voices that will be valuable in shaping future conversations. We describe the details of the initiative and the gaps we hope this project will help address.

Subfields: Artificial Intelligence, Information Systems and Management, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Percy Liang
3. Jianfeng Gao
4. Yoshua Bengio
5. Luke S. Zettlemoyer
6. Yejin Choi
7. Bo Li
8. Caiming Xiong

In Information Systems and Management

1. Adrian Weller
2. Andrew Smart
3. Ben Hutchinson
4. Peter Henderson
5. Emily L. Denton
6. A. Hanna
7. Margaret Mitchell
8. Sara Hooker

In Safety Research

1. Ben Hutchinson
2. Emily L. Denton
3. A. Hanna
4. F. Kroeger
5. Amanda Askell
6. Bo Li
7. Adrian Weller
8. Andrew Smart

Citations in the corpus

• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims (2020, background): “Inspiration might be taken from recent work on internal algorithmic auditing [36] and ongoing work on the documentation of AI systems more generally, such as the ABOUT ML project [27].”
• The Dataset Nutrition Label (2nd Gen): Leveraging Context to Mitigate Harms in Artificial Intelligence (2022, background): “In ethical AI documents, the principle of transparency is especially prevalent; other principles include justice, fairness, non-maleficence, responsibility, and privacy, among others [28, 22].”
• Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure (2020, background): “or team’s contribution to an overall system”, creating space for reflection, debate and recognition, and by giving “meaningful understanding of the impact of their personal participation” [116].”
• Trustworthy AI: From Principles to Practices (2021, background): “Some studies [22, 129, 156, 246, 273] have proposed standard practices to guide and evaluate the documentation of an AI system.”
• The Fallacy of AI Functionality (2022, unknown): “These efforts resemble the Partnership on AI’s attempt to establish norms on model documentation processes [153].”