Power Hungry Processing: Watts Driving the Cost of AI Deployment?

By Sasha Luccioni, Yacine Jernite, Emma Strubell. Conference on Fairness, Accountability and Transparency, 2023

461 citations recorded by the source corpus.

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of “generality” comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. In this work, we propose the first systematic comparison of the ongoing inference cost of various categories of ML systems, covering both task-specific (i.e. finetuned models that carry out a single task) and ‘general-purpose’ models, (i.e. those trained for multiple tasks). We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We find that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions. All the data from our study can be accessed via an interactive demo to carry out further exploration and analysis.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9eb476cd15becf02163d6f3dab75d207eed52214

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. L. K. Shar
2. Arda Goknil
3. E. J. Husom
4. Sagar Sen

In Computer Vision and Pattern Recognition

1. L. K. Shar
2. Arda Goknil
3. E. J. Husom
4. Sagar Sen

Citations in the corpus

• The Price of Prompting: Profiling Energy Use in Large Language Models Inference (2024, background): “Luccioni et al. [21] and Desislavov et al. [8] found that general-purpose ML models are more energy-intensive than task-specific ones and that efficiency gains in NLP and computer vision do not necessarily correlate with increased energy consumption.”