Robust artificial intelligence and robust human organizations
By Thomas G. Dietterich. 2018
36 citations recorded by the source corpus.
Every AI system is deployed by a human organization. In high risk applications, the combined human plus AI system must function as a high-reliability organization in order to avoid catastrophic errors. This short note reviews the properties of high-reliability organizations and draws implications for the development of AI technology and the safe application of that technology.
Subfields: Artificial Intelligence, Control and Systems Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Han-Jia Ye
- De-chuan Zhan
- Jacob Steinhardt
- Dan Hendrycks
- Percy Liang
- Nicholas Carlini
- Mantas Mazeika
- Dawn Song
In Control and Systems Engineering
- Dan Hendrycks
- Zhi-Hua Zhou
- Ben Shneiderman
- Jianxin Wu
- Han-Jia Ye
- Yuan Jiang
- Nicholas Carlini
- John Schulman
Citations in the corpus
- Multi-Instance Learning With Emerging Novel Class (2021, background): “However, both for binary or multi-class classification, it is noteworthy that most previous studies on multi-instance learning were in a stable environment rather than an environment that is open and changes gradually [13], [14].”
- Unsolved Problems in ML Safety (2021, background): “These complex and hazardous systems are now operated by high reliability organizations (HROs) which are relatively successful at avoiding catastrophes [47].”
- An Overview of Catastrophic AI Risks (2023, background): “Some organizations successfully avoid catastrophes while operating complex and hazardous systems such as nuclear reactors, aircraft carriers, and air traffic control systems [87, 88].”
- Heterogeneous Few-Shot Model Rectification With Semantic Mapping (2020, background): “AS MACHINE learning has been successfully applied to many real-world applications, whether a learner is robust or not has been attracting attentions [2], [3].”
- Bridging the Gap Between Ethics and Practice (2020, background): “More positive approaches were offered in descriptions of High Reliability Organizations [61, 12, 25], which emerged from organizational design and business administration, and Resilience Engineering [123, 44], which grew out of cognitive science and human factors engineering.”