Overtrust of robots in emergency evacuation scenarios

By P. Robinette, Wenchen Li, R. Allen, Ayanna M. Howard, Alan R. Wagner. 2016

424 citations recorded by the source corpus.

Subfields: Ocean Engineering, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Ocean Engineering

In Social Psychology

Citations in the corpus

Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition (2016, background): “These attacks are especially worrisome due to humans’ increasing reliance on technol-Permission ogy and ML (sometime trusting them even with their own lives [36]).”
Harms from Increasingly Agentic Algorithmic Systems (2023, background): “Similarly, Robinette et al. [2016] examines (over)trust of autonomous systems and find in emergency situations, people will follow robots into further danger, because they attribute the agency of the robot to the (assumed capable and responsible) designers.”
Towards a Theory of Longitudinal Trust Calibration in Human–Robot Teams (2019, background): “Trusting too much (“overtrust”) can condition operators into complacent states and misuse which can lead to costly disasters with the loss of human lives and destruction of expensive equipment [58,111,121,127].”
Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development (2018, methodology): “…et al., 2015; Law et al., 2017), future contact intensions with the robot (Short et al., 2010; Salem et al., 2013, 2015; Brooks et al., 2016; Robinette et al., 2016; Kwon et al., 2018), the robot being a good teammate (Kwon et al., 2018), psychological closeness with the robot (Salem et…”
Will You Accept an Imperfect AI?: Exploring Designs for Adjusting End-user Expectations of AI Systems (2019, background): “…their applications to behave inconsistently and imperfectly [15] which can lead to disap-pointment and potential abandonment of these technologies [41, 51] Previous work has shown that end-user expectations of technology are impacted by a variety of factors including external information,…”
Human Trust in Artificial Intelligence: Review of Empirical Research (2020, background): “Similarly, Robinette, Howard, and Wagner (2015) and Robinette, Li, Allen, Howard, and Wagner (2016) found that people tend to follow a robot ’ s lead in an emergency situation, even when its prior behavior indicates a serious malfunction.”
The Illusion of Control: Placebo Effects of Control Settings (2018, background): “It is already well-established that users default to considering automated systems trustworthy [16] and may be liable to overtrusting systems [39, 40, 49].”
How People Explain Action (and Autonomous Intelligent Systems Should Too) (2017, background): “People sometimes over-trust artificial agents, such as when they comply with a faulty robot’s unusual requests (Salem et al. 2015) or follow the lead of a potentially dysfunctional robot (Robinette et al. 2016).”
Helping Robots Learn: A Human-Robot Master-Apprentice Model Using Demonstrations via Virtual Reality Teleoperation (2020, background): “dencies of users to over-trust a supervised automation and inappropriately rely on it in multi-tasking scenarios [40] and particularly with embodied robots [41].”
Measurement of Trust in Automation: A Narrative Review and Reference Guide (2021, background): “Prior TiA studies has successfully introduced these characteristics, whether manipulating risk by threating virtual convoys with explosive attacks (Lyons and Stokes, 2012) or simulating escaping a burning building with a robot aid (Robinette et al., 2016).”