On a Formal Model of Safe and Scalable Self-driving Cars

By Shai Shalev-Shwartz, Shaked Shammah, Amnon Shashua. 2017

851 citations recorded by the source corpus.

In recent years, car makers and tech companies have been racing towards self driving cars. It seems that the main parameter in this race is who will have the first car on the road. The goal of this paper is to add to the equation two additional crucial parameters. The first is standardization of safety assurance --- what are the minimal requirements that every self-driving car must satisfy, and how can we verify these requirements. The second parameter is scalability --- engineering solutions that lead to unleashed costs will not scale to millions of cars, which will push interest in this field into a niche academic corner, and drive the entire field into a "winter of autonomous driving". In the first part of the paper we propose a white-box, interpretable, mathematical model for safety assurance, which we call Responsibility-Sensitive Safety (RSS). In the second part we describe a design of a system that adheres to our safety assurance requirements and is scalable to millions of cars.

Subfields: Artificial Intelligence, Automotive Engineering, Computational Theory and Mathematics, Safety, Robustness & Interpretability, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computational Theory and Mathematics

In Safety, Robustness & Interpretability

In Robotics & Control

Citations in the corpus

Scalable agent alignment via reward modeling: a research direction (2018, background): “Moreover, while reaching 100% trust in our systems is impossible, it is also not necessary: we only need to aim for a level of trust at which we can conﬁdently say that our new systems are more aligned than the current systems (Shalev-Shwartz et al., 2017).”
A Survey of State-Action Representations for Autonomous Driving (2018, methodology): “In (Shalev-Shwartz and Shashua, 2016; Shalev-Shwartz et al., 2016b) and (Shalev-Shwartz et al., 2017), the authors argue that due to this dense time resolution of decision making, the estimation of the value function faces a very small signal-to-noise ratio as its variance grows linearly with the…”
Actively Semi-Supervised Deep Rule-based Classifier Applied to Adverse Driving Scenarios (2019, background): “Therefore, mechanisms that can provide a guarantee of safety is of supreme importance to public acceptance of autonomous vehicles [18].”
Software Engineering for AI-Based Systems: A Survey (2021, background): “Five papers focus on formal model analysis with different goals: analysis of safety and scalability in models for autonomous vehicles [183], quantitative analysis for systems based on recurrent neural networks [49], improvement and certification of robustness for ML models [220], and approaches for ”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, background): “However, the learning-based nature of end-to-end frameworks inherently lacks precise mathematical guarantees regarding safety, unlike traditional rule-based approaches [237].”
VALUATION : A N ADVERSARIAL APPROACH TO UNCOVER CATASTROPHIC FAILURES (2018, positive): “Our aims share similarities to recent proposals for testing components of autonomous vehicle systems (Shalev-Shwartz et al., 2017; Dreossi et al., 2017; Tian et al., 2018).”