Intelligent Agent Transparency in Human–Agent Teaming for Multi-UxV Management

By Joseph E. Mercado, Michael A. Rupp, Jessie Y.C. Chen, Michael J. Barnes, Daniel J. Barber, Katelyn Procci. 2016

389 citations recorded by the source corpus.

Objective: We investigated the effects of level of agent transparency on operator performance, trust, and workload in a context of human–agent teaming for multirobot management. Background: Participants played the role of a heterogeneous unmanned vehicle (UxV) operator and were instructed to complete various missions by giving orders to UxVs through a computer interface. An intelligent agent (IA) assisted the participant by recommending two plans—a top recommendation and a secondary recommendation—for every mission. Method: A within-subjects design with three levels of agent transparency was employed in the present experiment. There were eight missions in each of three experimental blocks, grouped by level of transparency. During each experimental block, the IA was incorrect three out of eight times due to external information (e.g., commander’s intent and intelligence). Operator performance, trust, workload, and usability data were collected. Results: Results indicate that operator performance, trust, and perceived usability increased as a function of transparency level. Subjective and objective workload data indicate that participants’ workload did not increase as a function of transparency. Furthermore, response time did not increase as a function of transparency. Conclusion: Unlike previous research, which showed that increased transparency resulted in increased performance and trust calibration at the cost of greater workload and longer response time, our results support the benefits of transparency for performance effectiveness without additional costs. Application: The current results will facilitate the implementation of IAs in military settings and will provide useful data to the design of heterogeneous UxV teams.

Subfields: Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Social Psychology

Citations in the corpus

Explanation in Artificial Intelligence: Insights from the Social Sciences (2017, background): “Applications of explanation are considered in many sub-fields of artificial intelligence, such as justifying autonomous agent behaviour [129, 65], debugging of machine learning models [89], explaining medical decision-making [45], and explaining predictions of classifiers [156].”
Towards a Theory of Longitudinal Trust Calibration in Human–Robot Teams (2019, methodology): “These methods revolve around the central method of increasing the transparency of a system [13], such as its performance, its process, and its intent and purpose [3,13,14,99].”
Implementing Lumberjacks and Black Swans Into Model-Based Tools to Support Human–Automation Interaction (2017, background): “• Capitalize on the meaning of the “loss of SA curve” and carefully craft intuitive displays of dynamic processes that can improve SA without imposing significant mental effort on the operator, in the spirit of ecological displays (Burns et al., 2008; Hoff & Bashir, 2015; Mercado et al., 2016).”
Explanations as Mechanisms for Supporting Algorithmic Transparency (2018, background): “Explanations improve system usability and overall performance, and promote more positive user perceptions and acceptance of the system [10, 24, 21, 37].”
From ‘automation’ to ‘autonomy’: the importance of trust repair in human–machine interaction (2018, positive): “Importantly, there should be enough transparency to support and foster trust calibration (Chen et al. 2014; Mercado et al. 2016; Zuk and Carpendale 2007).”
Situation awareness-based agent transparency and human-autonomy teaming effectiveness (2018, methodology): “Mercado et al. (2016) found that when interacting with an intelligent planning agent, operator performance and trust in the agent increased as a function of agent transparency level.”
Transparency and Explanation in Deep Reinforcement Learning Neural Networks (2018, methodology): “This new model can be incorporated with most existing deep RL frameworks such as DQN (Mnih et al. 2015) and A3C (Mnih et al. 2016).”
Transparency in artificial intelligence (2020, background): “The following hypothesis is then that by building more explainable systems, users would be better equipped to understand and thereby trust the intelligent agents or predictive modelling (Mercado et al., 2016; see also Kopitar, Cilar, Kocbek, & Stiglic, 2019, for a medical example).”
Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature (2020, background): “…cooperatively with at least one autonomous agent (McNeese et al., 2018), where an autonomous agent is a computer entity with a partial or high degree of self- governance with respect to decision- making, adaptation, and communication (Demir et al., 2016; Mercado et al., 2016; Myers et al., 2019).”
Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence (2021, background): “…the development of explainable AI systems is motivated by several social and ethical reasons (see, for example, [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]): increasing societal acceptance of prediction-based decisions, establishing trust in the results of these decisions, making algorithms…”