Plan Explanations as Model Reconciliation: Moving Beyond Explanation as Soliloquy

By Tathagata Chakraborti, Sarath Sreedharan, Yu Zhang, Subbarao Kambhampati. 2017

349 citations recorded by the source corpus.

When AI systems interact with humans in the loop, they are often called on to provide explanations for their plans and behavior. Past work on plan explanations primarily involved the AI system explaining the correctness of its plan and the rationale for its decision in terms of its own model. Such soliloquy is wholly inadequate in most realistic scenarios where the humans have domain and task models that differ significantly from that used by the AI system. We posit that the explanations are best studied in light of these differing models. In particular, we show how explanation can be seen as a "model reconciliation problem" (MRP), where the AI system in effect suggests changes to the human's model, so as to make its plan be optimal with respect to that changed human model. We will study the properties of such explanations, present algorithms for automatically computing them, and evaluate the performance of the algorithms.

Subfields: Artificial Intelligence, Ocean Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Ocean Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “Other introductory works include [171] and [172], in addition of a dozen papers that was discussed very recently in a dedicated session of the ICAPS Workshop [11].”
Explain Yourself: A Natural Language Interface for Scrutable Autonomous Robots (2018, background): “In previous work, explanations have been categorised as either explaining 1) machine learning as in [11] who showed that they can increase trust; 2) explaining plans [2, 13]; 3) verbalising robot [12] or agent rationalisation [3].”
Explainable Planning (2017, background): “In Model reconciliation [Chakraborti et al., 2017], the focus is on the agent and the human having two different models, hence the explanations must identify and reconcile the relevant differences between the models.”
Explanation in Artificial Intelligence: Insights from the Social Sciences (2017, background): “[21] discuss preliminary work in this area for ex-”
Handling Model Uncertainty and Multiplicity in Explanations via Model Reconciliation (2018, methodology): “A solution to an MRP (Chakraborti et al. 2017) is the set of model changes E or an explanation , such that the given plan π is optimal in both the robot model M R and the updated human model M RH .”
A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence (2021, methodology): “Moreover, some of the robotics-related frameworks found in reinforcement learning settings provide explanations for policies that a robot selects at a given time step [132], [156].”
Explainable Autonomy: A Study of Explanation Styles for Building Clear Mental Models (2018, background): “…models for developers, as in (Ribeiro et al., 2016) who showed that such explanations can increase trust; 2) explanations of planning systems (Tintarev and Kutlak, 2014; Chakraborti et al., 2017); and 3) verbalis-ing robot (Rosenthal et al., 2016) or agent (Harrison et al., 2017) rationalisation.”
A Grounded Interaction Protocol for Explainable Artificial Intelligence (2019, background): “nation ratherbasingtheapproachonastrongunderstandingofhowpeople define, generate, select, evaluate, and present explanations [21, 22]. Most modern work on Explainable AI, such as in autonomous agents [5, 7, 15, 36] and interpretable machine learning [13], does not discuss the interaction and the soc”
Towards Explainable Shared Control using Augmented Reality (2019, background): “Model misalignment or “model reconciliation” [11] is a well-known problem that has been explored in a related 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019”
Explainable Agents and Robots: Results from a Systematic Literature Review (2019, neutral): “Intent communication [3] [98] [85] [13] [70] [25] [92] [93] [31] [78] [17] [52]”
Algorithms for the Greater Good! On Mental Modeling and Acceptable Symbiosis in Human-AI Collaboration (2018, background): “Finally, I note that all the use cases covered in the paper are, in fact, borne directly out of technologies or algorithms that I have developed (Chakraborti et al. 2015; Chakraborti et al. 2017b), albeit with slight modiﬁcations. as a student researcher over the last couple of years.”
Model Recognition as Planning (2019, methodology): “…to conform two PDDL models, where one of them can be an annotated model or an empty model, with respect to a fully observed optimal plan computed with one of the two models (Chakraborti et al. 2017; Sreedharan, Chakraborti, and Kambhampati 2018; Chakraborti, Sreedharan, and Kambhampati 2018).”
Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systems (2021, positive): “Explanations via Symbolic Interface: Prior work in explainable AI systems has already noted that explanations cannot be a soliloquy, and must be in terms of the vocabulary and mental models of the recipients (Chakraborti et al. 2017).”
Hierarchical Expertise-Level Modeling for User Specific Robot-Behavior Explanations (2018, background): “Another approach that has received a lot of interest recently is to view explanations as a way of achieving model reconciliation[Chakraborti et al., 2017].”
Explanations in Autonomous Driving: A Survey (2021, background): “The explainable planning frameworks described above and the related work by [90], [137], [139], [140] can serve as”