Explainable Agency for Intelligent Autonomous Systems

By Pat Langley, B. Meadows, M. Sridharan, Dongkyu Choi. 2017

198 citations recorded by the source corpus.

As intelligent agents become more autonomous, sophisti- cated, and prevalent, it becomes increasingly important that humans interact with them effectively. Machine learning is now used regularly to acquire expertise, but common techniques produce opaque content whose behavior is difficult to interpret. Before they will be trusted by humans, autonomous agents must be able to explain their decisions and the reasoning that produced their choices. We will refer to this general ability as explainable agency. This capacity for explaining decisions is not an academic exercise. When a self-driving vehicle takes an unfamiliar turn, its passenger may desire to know its reasons. When a synthetic ally in a computer game blocks a player’s path, he may want to understand its purpose. When an autonomous military robot has abandoned a high-priority goal to pursue another one, its commander may request justification. As robots, vehicles, and synthetic characters become more selfreliant, people will require that they explain their behaviors on demand. The more impressive these agents’ abilities, the more essential that we be able to understand them.

Subfields: Artificial Intelligence, LLM Agents & Tool Use, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Longo Luca
  2. Giulia Vilone
  3. Derek Long
  4. Maria Fox
  5. Daniele Magazzeni
  6. Christian Walter Peter Omlin
  7. Waddah Saeed
  8. Chunyan Miao

In LLM Agents & Tool Use

  1. Derek Long
  2. Maria Fox
  3. Victor R. Lesser
  4. Chongjie Zhang

In Safety, Robustness & Interpretability

  1. Longo Luca
  2. Giulia Vilone
  3. Christian Walter Peter Omlin
  4. Waddah Saeed
  5. Qiang Yang
  6. Mohammed Berrada
  7. Amina Adadi
  8. Gesina Schwalbe

Citations in the corpus