Towards Verified Artificial Intelligence

By Sanjit A. Seshia, Dorsa Sadigh. 2016

211 citations recorded by the source corpus.

Verified artificial intelligence (AI) is the goal of designing AI-based systems that that have strong, ideally provable, assurances of correctness with respect to mathematically-specified requirements. This paper considers Verified AI from a formal methods perspective. We describe five challenges for achieving Verified AI, and five corresponding principles for addressing these challenges.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Software

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Software

Citations in the corpus

AI Safety Gridworlds (2017, background): “, 2017), formal verification (Seshia et al., 2016; Huang et al., 2017b; Katz et al., 2017), scalable oversight and reward learning problems (Amodei et al.”
Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262 (2018, background): “Some early proposals on doing formal verification of ML models can be found in [68].”
Trustworthy AI: From Principles to Practices (2021, unknown): “Simulation [42, 50, 54, 103, 192, 205, 323, 324, 359] Formal Verification [164, 298, 335]”
Software Engineering for AI-Based Systems: A Survey (2021, unknown): “Two papers [86, 183] present an overview of current challenges and practices in V&V of autonomous systems, in particular those based on DL.”
Planning and Decision-Making for Autonomous Vehicles (2018, background): “(135) identified five challenges: modeling of the complex environment, modeling of the system, formal specification of the desired properties of the system, scalability, and formal quantification of the requirements for training data.”
Teaching Temporal Logics to Neural Networks (2020, methodology): “Another area in the intersection of formal methods and machine learning is the verification of neural networks [41,40,43,12,18,8].”
Will Distributed Computing Revolutionize Peace? The Emergence of Battlefield IoT (2018, unknown): “Recent work [37] describes challenges for applying”