Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models

By Lasse F. Wolff Anthony, Benjamin Kanding, Raghavendra Selvan. 2020

433 citations recorded by the source corpus.

Deep learning (DL) can achieve impressive results across a wide variety of tasks, but this often comes at the cost of training models for extensive periods on specialized hardware accelerators. This energy-intensive workload has seen immense growth in recent years. Machine learning (ML) may become a significant contributor to climate change if this exponential trend continues. If practitioners are aware of their energy and carbon footprint, then they may actively take steps to reduce it whenever possible. In this work, we present Carbontracker, a tool for tracking and predicting the energy and carbon footprint of training DL models. We propose that energy and carbon footprint of model development and training is reported alongside performance metrics using tools like Carbontracker. We hope this will promote responsible computing in ML and encourage research into energy-efficient deep neural networks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In AI Ethics, Fairness & Society

Citations in the corpus

Measuring the Carbon Intensity of AI in Cloud Instances (2022, methodology): “This is a more straightforward endeavor compared to other stages both upstream and downstream from the training process, given that it is well-defined in time and its emissions can be measured in real-time with tools like Code Carbon [38] and Carbon Tracker [1] or estimated post-hoc using tools such”
The AI gambit: leveraging artificial intelligence to combat climate change—opportunities, challenges, and recommendations (2021, background): “…electrical grid (Schwartz et al. 2019; Anthony et al. 2020), the number of experiments required during model construction (Schwartz et al. 2019; Strubell et al. 2019), and the time period in which a model was trained, as carbon/emission intensity can vary throughout the day (Anthony et al. 2020) .”
Metrics Reloaded: Recommendations for image analysis validation (2022, neutral): “A.T. was an active member of the calibration group, reviewed the manuscript, and participated in surveys.”
Unraveling the hidden environmental impacts of AI solutions for environment (2021, methodology): “• Integrated tools, such as Experiment Impact Tracker 1 [17], Carbon Tracker 2 [2] and CodeCarbon 3, which”
Aligning artificial intelligence with climate change mitigation (2022, neutral): “Modern ML models that use neural networks are particularly energy intensive in the development phase as they have many more possible model configurations than their predecessors, and it is not”
Sustainable AI: AI for sustainability and the sustainability of AI (2021, methodology): “More to the point, “the carbon emissions that occur when training DL models are not irreducible and do not have to simply be the cost of progress within DL” [3, 3].”
A systematic review of Green AI (2023, methodology): “As an example, the Carbontracker offers a toolset to track and predict the energy and carbon foot-print of training DL models (Wolff Anthony et al., 2020).”
Training Spiking Neural Networks Using Lessons From Deep Learning (2021, background): “OpenAI’s language model, GPT-3, contains 175 billion learnable parameters, estimated to consume roughly 190 000 kWh to train [25], [26], [27].”
EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models (2021, methodology): “[3] builds on prior work by ofering predictions for total carbon consumption through a command line interface.”
A first look into the carbon footprint of federated learning (2020, background): “Existing results (Lacoste et al., 2019; Anthony et al., 2020) propose approaches that consider CO2e emissions only under centralized training assumptions.”
Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model (2022, methodology): “[30, 1]), while others can be used post-training in order to produce a more high-level estimate of emissions (e.”
The Price of Prompting: Profiling Energy Use in Large Language Models Inference (2024, methodology): “Tools like Green Algorithms [16], CodeCarbon [6], and CarbonTracker [3] have been developed to estimate computational carbon emissions, yet they often provide general rather than detailed analyses necessary for large-scale LLM deployment.”