The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research

By N. Ahmed, Muntasir Wahed. 2020

162 citations recorded by the source corpus.

Increasingly, modern Artificial Intelligence (AI) research has become more computationally intensive. However, a growing concern is that due to unequal access to computing power, only certain firms and elite universities have advantages in modern AI research. Using a novel dataset of 171394 papers from 57 prestigious computer science conferences, we document that firms, in particular, large technology firms and elite universities have increased participation in major AI conferences since deep learning's unanticipated rise in 2012. The effect is concentrated among elite universities, which are ranked 1-50 in the QS World University Rankings. Further, we find two strategies through which firms increased their presence in AI research: first, they have increased firm-only publications; and second, firms are collaborating primarily with elite universities. Consequently, this increased presence of firms and elite universities in AI research has crowded out mid-tier (QS ranked 201-300) and lower-tier (QS ranked 301-500) universities. To provide causal evidence that deep learning's unanticipated rise resulted in this divergence, we leverage the generalized synthetic control method, a data-driven counterfactual estimator. Using machine learning based text analysis methods, we provide additional evidence that the divergence between these two groups - large firms and non-elite universities - is driven by access to computing power or compute, which we term as the "compute divide". This compute divide between large firms and non-elite universities increases concerns around bias and fairness within AI technology, and presents an obstacle towards "democratizing" AI. These results suggest that a lack of access to specialized equipment such as compute can de-democratize knowledge production.

Subfields: Artificial Intelligence, Health Informatics, Safety Research, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Health Informatics

In Safety Research

In Scaling & Learning Theory

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Specific resources such as computing power [16] are popularly tracked by annual reports on the field of AI [145, 27].”
The Values Encoded in Machine Learning Research (2021, methodology): “Furthermore, we also find a much greater concentration of a few large tech firms, such as Google and Microsoft, with the presence of these "big tech" firms (as identified in [4]) increasing more than fivefold, from 11% to 58% (see Figure 2).”
The AI gambit: leveraging artificial intelligence to combat climate change—opportunities, challenges, and recommendations (2021, methodology): “Neverthe-less, we note that research exploring new neural network architectures or new hardware – software – algorithm combinations has largely been side-lined in favour of compute-intensive AI research (Hooker 2020; Marcus 2020; Ahmed and Wahed 2020).”
The Fallacy of AI Functionality (2022, background): “have access to the ability to build these systems [8].”