The social dilemma of autonomous vehicles

By Jean‐François Bonnefon, Azim Shariff, Iyad Rahwan. 2015

1,338 citations recorded by the source corpus.

Codes of conduct in autonomous vehicles When it becomes possible to program decision-making based on moral principles into machines, will self-interest or the public good predominate? In a series of surveys, Bonnefon et al. found that even though participants approve of autonomous vehicles that might sacrifice passengers to save others, respondents would prefer not to ride in such vehicles (see the Perspective by Greene). Respondents would also not approve regulations mandating self-sacrifice, and such regulations would make them less willing to buy an autonomous vehicle. Science, this issue p. 1573; see also p. 1514 Programming an acceptable morality into driverless cars presents large challenges. Autonomous vehicles (AVs) should reduce traffic accidents, but they will sometimes have to choose between two evils, such as running over pedestrians or sacrificing themselves and their passenger to save the pedestrians. Defining the algorithms that will help AVs make these moral decisions is a formidable challenge. We found that participants in six Amazon Mechanical Turk studies approved of utilitarian AVs (that is, AVs that sacrifice their passengers for the greater good) and would like others to buy them, but they would themselves prefer to ride in AVs that protect their passengers at all costs. The study participants disapprove of enforcing utilitarian regulations for AVs and would be less willing to buy such an AV. Accordingly, regulating for utilitarian algorithms may paradoxically increase casualties by postponing the adoption of a safer technology.

Subfields: Accounting, Management Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Accounting

In Management Information Systems

In Management Science and Operations Research

Citations in the corpus

How Do Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness (2018, background): “In line with recent work examining public attitudes of the ethical programming of machines [4, 7], we suggest that these public attitudes serve as a useful and important input in a conversation between technologists and ethicists.”
A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “In an inevitable accident situation, how should the system behave [40]? Experimental ethics were proposed regarding this issue [41].”
Ethical governance is essential to building trust in robotics and artificial intelligence systems (2018, background): “The study participants disapprove of enforcing utilitarian regulations for AVs and would be less willing to buy such an AV’ [30].”
Accountable Artificial Intelligence: Holding Algorithms to Account (2020, background): “They are also anticipated to next encode ethical life-and-death decisions (e.g., in case of a crash, should a self-driving car prioritize saving its driver/passengers or pedestrians; Bonnefon, Shariff, and Rahwan 2016).”
Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys (2023, background): “between alternatives that will inevitably result in the sacrifice of human lives [151], [152], [153].”
Veil-of-ignorance reasoning favors the greater good (2019, background): “While previous research indicates net disapproval of utilitarian regulation of AVs (19), here we find that VOI reasoning shifts approval to as high as 83% (Fig.”
Human Decision Making with Machine Assistance (2019, background): “Several studies also explored human perceptions of the fairness of machine decisions, in the context of self-driving cars [5, 12], targeted-advertising [59], loan approval [62], and bail decisions [36–38 Another series of studies has compared human and machine decision-making.”
Building Ethics into Artificial Intelligence (2018, background): “…: technical systems en-Table [Anderson and Anderson, 2014] [Dehghani et al. , 2008] [Singh, 2014; Singh, 2015] [Battaglino and Damiano, 2015] [Bonnefon et al. , 2016] [Blass and Forbus, 2015] [Pagallo, 2016] [Stock et al. , 2016] [Sharif et al. , 2017] [van Riemsdijk et al. , 2015] [Greene…”
The Trolley, the Bull Bar, and Why Engineers Should Care About the Ethics of Autonomous Cars (2019, background): “It is also useful for assessing people’s moral intuitions without burdening them with an intricate mechanical description of just how such an improbable situation might arise [5], [6].”
Gender differences in beliefs about algorithmic fairness (2017, background): “Previous work has shown that people’s moral judgments correlate with their demographic traits [5, 6], as do beliefs about the ethics of self-driving cars [2] and beliefs about fairness and discrimination [10, 12].”