Strategic Implications of Openness in AI Development

By Nick Bostrom. 2017

119 citations recorded by the source corpus.

This paper attempts a preliminary analysis of the global desirability of different forms of openness in AI development (including openness about source code, science, data, safety techniques, capabilities, and goals). Short-term impacts of increased openness appear mostly socially beneficial in expectation. The strategic implications of medium and long-term impacts are complex. The evaluation of long-term impacts, in particular, may depend on whether the objective is to benefit the present generation or to promote a time-neutral aggregate of well-being of future generations. Some forms of openness are plausibly positive on both counts (openness about safety measures, openness about goals). Others (openness about source code, science, and possibly capability) could lead to a tightening of the competitive situation around the time of the introduction of advanced AI, increasing the probability that winning the AI race is incompatible with using any safety method that incurs a delay or limits performance. We identify several key factors that must be taken into account by any well-founded opinion on the matter.

Subfields: Computer Science Applications, Management Science and Operations Research, Safety Research, AI Ethics, Fairness & Society, Philosophy

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Management Science and Operations Research

In Safety Research

In AI Ethics, Fairness & Society

In Philosophy

Citations in the corpus

A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy (2017, background): “In contrast, Bostrom (2017) argues that open-source AGI R&D could be more dangerous by giving everyone access to the same code and thereby tightening the race to build AGI first.”
The Role of Cooperation in Responsible AI Development (2019, methodology): “This means that a company that is ahead of others may at some point be ahead of them by a great deal in strategically important areas, and could use this to undermine their competitors (Bostrom, 2017b, pp. 91-104) (9)Breitmoser et al. (2010) note that perpetual R&D races tend to collapse into leader”
Limits and Possibilities for “Ethical AI” in Open Source: A Study of Deepfakes (2022, background): “The risk of this openness aiding the proliferation of potientially harmful technology such as superhuman AI [15], and claims that open source contributors are unacceptably expected to abrogate control over the ethical impact of their creations [87] have been explored before, and we unpack how open s”