Transparency and trust in artificial intelligence systems

By Philipp Schmidt, Felix Biessmann, Timm Teubner. Journal of Decision Systems, 2020

342 citations recorded by the source corpus.

ABSTRACT Assistive technology featuring artificial intelligence (AI) to support human decision-making has become ubiquitous. Assistive AI achieves accuracy comparable to or even surpassing that of human experts. However, often the adoption of assistive AI systems is limited by a lack of trust of humans into an AI’s prediction. This is why the AI research community has been focusing on rendering AI decisions more transparent by providing explanations of an AIs decision. To what extent these explanations really help to foster trust into an AI system remains an open question. In this paper, we report the results of a behavioural experiment in which subjects were able to draw on the support of an ML-based decision support tool for text classification. We experimentally varied the information subjects received and show that transparency can actually have a negative impact on trust. We discuss implications for decision makers employing assistive AI technology.

Subfields: Artificial Intelligence, Safety Research, Social Psychology

View Paper: https://www.semanticscholar.org/paper/7a19f30e02c34c4eb7b197d3bcd4fbcb8a4e1602

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Q. Liao
2. Jennifer Wortman Vaughan
3. Arun Ross
4. Prabu David
5. Hyesun Choung
6. Ming Yin

In Safety Research

1. Q. Liao
2. Paul Rodway
3. Astrid Schepman
4. Jennifer Wortman Vaughan
5. Arun Ross
6. Prabu David
7. Hyesun Choung

In Social Psychology

1. Q. Liao
2. Arun Ross
3. Paul Rodway
4. Astrid Schepman
5. Jennifer Wortman Vaughan
6. Prabu David
7. Hyesun Choung

Citations in the corpus

• The General Attitudes towards Artificial Intelligence Scale (GAAIS): Confirmatory Validation and Associations with Personality, Corporate Distrust, and General Trust (2022, background): “A further aspect of trust is linked to explainability (see e.g., EU, 2020; Schmidt et al., 2020; Stanton & Jensen, 2021).”
• Trust in AI: progress, challenges, and future directions (2024, neutral): “For example, researchers have shown through a behavioral experiment that giving excessive transparency will confuse the user and negatively impact trust (Schmidt et al., 2020a; 2020b).”
• Trust in AI and Its Role in the Acceptance of AI Technologies (2022, background): “We expected that both dimensions of trust would influence technology adoption and use (Schmidt et al., 2020), especially for technologies that are associated with risk and uncertainty.”
• AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, result): “Studies of other transparency approaches have also reported nuanced results [160, 167, 204, 208].”