Trust in Automation: Designing for Appropriate Reliance

By John D. Lee, Katrina A. See. 2004

6,102 citations recorded by the source corpus.

Subfields: Experimental and Cognitive Psychology, Information Systems and Management, Social Psychology, AI Ethics, Fairness & Society, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Information Systems and Management

In Social Psychology

In AI Ethics, Fairness & Society

In Human-Computer Interaction

Citations in the corpus

Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI (2019, background): “…Miller, 2011; Hoffman, 2017a; Jian, Bisantz, and Drury, 1998, 2000; Johnson, 2007; Koustanai, Mas, Cavallo, and Delhomme, 2010; Lee and Moray, 1994; Lee and See, 2004; Llinas, Bisantz, Drury, Seong, and Jian, 1998; Lyons et al., 2017; Merritt, 2011; Merritt, Heimbaugh, LaChapell, and Lee, 2013;…”
Metrics for Explainable AI: Challenges and Prospects (2018, neutral): “…(e.g., Chancey et al., 2015; Hoff and Bashir 2015; Fitzhugh, Hoffman & Miller, 2011; Hoffman et al., 2009, 2013; Huynh et al., 2006; Kucala, 2013; Lee & See, 2004; Merritt and Ilgen, 2008; Merritt et al. 2013, 2015; Naone, 2009; Pop et al. 2015; Shadbolt, 2002; Wickens et al., 2015; Woods and…”
Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI (2020, background): “16,19 Trust is well calibrated when the human sets their trust level appropriately to the AI’s capabilities, accepting the output of a competent system but employing other resources or their own expertise to compensate for AI errors; conversely, poorly calibrated trust reduces team performance…”
In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes (2019, methodology): “In this paper, we adopt a reconciliation of the conflicting definitions of trust proposed by Lee and See [24].”
Improving Robot Controller Transparency Through Autonomous Policy Explanation (2017, methodology): “Automated methods utilizing natural language-based explanations (rather than source code analysis) have been effective for instilling appropriate levels of understanding and confidence regarding expected robot behavior [22].”
To Trust Or Not To Trust A Classifier (2018, background): “As such, ML trust and safety is an important theme [1, 2, 3].”
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI (2020, background): “ay that the trust is warranted if it is the result of trustworthiness, and otherwise it is unwarranted [50]. Warranted trust is sometimes referred to as trust which is calibrated with trustworthiness [43]. In other words, trust is the cognitive mechanism to give the ‘illusion’ of anticipating intend”
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance (2019, background): “In fact, appropriate trust in automation is a topic that has received early attention (Lee and See 2004) as determinant factor for designing systems that require people to manage and intervene during imperfect automation.”
Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians (2019, background): “The development of AI must incorporate mechanisms that will establish and maintain a properly balanced, optimal level of trust from and to the user that matches the capability of the AI system [12].”
Toward a framework for levels of robot autonomy in human-robot interaction (2014, background): “A number of models and theories related to trust in automation (Cohen, Parasuraman, & Freeman, 1998; Dzindolet et al., 2003; Lee & See, 2004; Madhavan & Wiegmann, 2007) and preliminary frameworks of trust in HRI have been proposed (Desai, Stubbs, Steinfeld, & Yanco, 2009; Hancock, Billings, &…”
Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving (2013, background): “Studies from other domains, such as the aviation domain, have shown that the anticipated positive effects of automation might be diminished due to human-automation cooperation related problems, such as automation misuse and disuse [1, 2], automation surprises and mode confusion [3, 4], reduced situa”
M‐payment service: Interplay of perceived risk, benefit, and trust in service adoption (2018, background): “On human–technology relationships, trust is a critical factor that affects system effectiveness, especially as it relates to safety, performance, and usage rate (J. D. Lee & See, 2004).”
Similarities and differences between human–human and human–automation trust: an integrative review (2007, result): “On the whole, the findings by Lerch et al. (1997) revealed that (1) participants had lower confidence in advice from an expert system than from a human expert, (2) participants rated ‘effort’ as contributing less to the performance of an expert system than a human expert and (3) while human advisors”
Individual Differences in the Calibration of Trust in Automation (2015, methodology): “A large body of research has identified that one of the critical variables that may influence decisions about automation use is an operator’s trust in the automation (see Lee & See, 2004 for a review).”
Strategies for human-in-the-loop robotic grasping (2012, result): “” In general, our results are consistent with previous work on building trust between an autonomous system and an operator [10].”
The Perception And Measurement Of Human-robot Trust (2013, background): “The relationship among calibration, resolution, and automation capability in defining appropriate trust (Lee & See, 2004).”