Quantifying the Carbon Emissions of Machine Learning

By Alexandre Lacoste, Alexandra Sasha Luccioni, Victor Schmidt, Thomas Dandres. 2019

970 citations recorded by the source corpus.

From an environmental standpoint, there are a few crucial aspects of training a neural network that have a major impact on the quantity of carbon that it emits. These factors include: the location of the server used for training and the energy grid that it uses, the length of the training procedure, and even the make and model of hardware on which the training takes place. In order to approximate these emissions, we present our Machine Learning Emissions Calculator, a tool for our community to better understand the environmental impact of training ML models. We accompany this tool with an explanation of the factors cited above, as well as concrete actions that individual practitioners and organizations can take to mitigate their carbon emissions.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

Efficient Large Scale Language Modeling with Mixtures of Experts (2021, methodology): “13Carbon efficiency is estimated using the Machine Learning Impact calculator (Lacoste et al., 2019) and the PUE estimate is based on Bergman (2021). kgCO2e of emissions, of which 100 percent is directly offset by the cloud provider.”
Carbon Emissions and Large Neural Network Training (2021, methodology): “Without access to such a dashboard, use the ML Emissions Calculator [Lac19] or Green Algorithms tool [Lan20] that estimate the CO 2 e mix by region (see Figure 6 below) 11 .”
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, background): “Apart from environmental concerns (Strubell et al., 2019; Lacoste et al., 2019; Schwartz et al., 2020), the costs of training large language models (LLMs) are only affordable for well-resourced organizations.”
GPT-NeoX-20B: An Open-Source Autoregressive Language Model (2022, background): “A significant point of concern in some recent work is the energy usage and carbon emissions associated with training large language models (Strubell et al., 2019; Schwartz et al., 2020; Lacoste et al., 2019; Bender et al., 2021).”
Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “Despite the recent calls-to-action [27], [37], [38], [39], [21], the overall community remains under-invested in research that aims at deeply understanding and minimizing the cost of AI.”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “For instance, [132] proposes both a web-based and programmatic approach for quantifying the carbon emissions of models, meanwhile [104] proposes an experiment-impact-tracker, for energy and carbon usage reporting research.”
Measuring the Carbon Intensity of AI in Cloud Instances (2022, methodology): “This is a more straightforward endeavor compared to other stages both upstream and downstream from the training process, given that it is well-defined in time and its emissions can be measured in real-time with tools like Code Carbon [38] and Carbon Tracker [1] or estimated post-hoc using tools such”
StarCoder: may the source be with you! (2023, neutral): “StarCoderBase We report the carbon footprint (Lacoste et al., 2019)”
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, background): “This could incur high financial [26] and environmental [33] costs if the replacement is frequent.”
Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning (2020, background): “Lacoste et al. (2019) recently provided a website for estimating carbon emissions based on GPU type, experiment length, and cloud provider.”
SECure: A Social and Environmental Certificate for AI Systems (2020, background): “Lacoste et al. (2019) and Henderson et al. (2020) engage critically with cloud providers’ claims about carbon neutrality.”
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining (2023, background): “Large language models are We hope to improve training efﬁciency and reduce the environmental impact of training large LMs [50; 28; 37; 29].”