Interactive Model Cards: A Human-Centered Approach to Model Documentation

By Anamaria Crisan, Margaret Drouhard, Jesse Vig, Nazneen Rajani. 2022

143 citations recorded by the source corpus.

Deep learning models for natural language processing (NLP) are increasingly adopted and deployed by analysts without formal training in NLP or machine learning (ML). However, the documentation intended to convey the model’s details and appropriate use is tailored primarily to individuals with ML or NLP expertise. To address this gap, we conduct a design inquiry into interactive model cards, which augment traditionally static model cards with affordances for exploring model documentation and interacting with the models themselves. Our investigation consists of an initial conceptual study with experts in ML, NLP, and AI Ethics, followed by a separate evaluative study with non-expert analysts who use ML models in their work. Using a semi-structured interview format coupled with a think-aloud protocol, we collected feedback from a total of 30 participants who engaged with different versions of standard and interactive model cards. Through a thematic analysis of the collected data, we identified several conceptual dimensions that summarize the strengths and limitations of standard and interactive model cards, including: stakeholders; design; guidance; understandability & interpretability; sensemaking & skepticism; and trust & safety. Our findings demonstrate the importance of carefully considered design and interactivity for orienting and supporting non-expert analysts using deep learning models, along with a need for consideration of broader sociotechnical contexts and organizational dynamics. We have also identified design elements, such as language, visual cues, and warnings, among others, that support interactivity and make non-interactive content accessible. We summarize our findings as design guidelines and discuss their implications for a human-centered approach towards AI/ML documentation.

Subfields: Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Human-Computer Interaction

1. Jonas Schuett
2. Q. Liao
3. Luciano Floridi
4. Jakob Mokander
5. Hannah Rose Kirk
6. Mika Westerlund

Citations in the corpus

• The Gradient of Generative AI Release: Methods and Considerations (2023, background): “Model cards are deployed across Hugging Face’s platform and have evolved to be interactive [26].”
• Auditing large language models: a three-layered approach (2023, background): “, interactive model cards [183] and information about the initial training dataset [184], [185], to help downstream developers adapt the model appropriately.”
• AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “Following recent HCI and FATE studies investigating how to design effective documentation interfaces [38, 121], we suggest that those designing model reports for LLMs should explore more interactive features.”