Evaluating the carbon footprint of NLP methods: a survey and analysis of existing tools
By Nesrine Bannour, Sahar Ghannay, Aurélie Névéol, Anne-Laure Ligozat. 2021
111 citations recorded by the source corpus.
Modern Natural Language Processing (NLP) makes intensive use of deep learning methods because of the accuracy they offer for a variety of applications. Due to the significant environmental impact of deep learning, cost-benefit analysis including carbon footprint as well as accuracy measures has been suggested to better document the use of NLP methods for research or deployment. In this paper, we review the tools that are available to measure energy use and CO2 emissions of NLP methods. We describe the scope of the measures provided and compare the use of six tools (carbon tracker, experiment impact tracker, green algorithms, ML CO2 impact, energy usage and cumulator) on named entity recognition experiments performed on different computational set-ups (local server vs. computing facility). Based on these findings, we propose actionable recommendations to accurately measure the environmental impact of NLP experiments.
Subfields: Artificial Intelligence, Management Science and Operations Research, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Mohit Bansal
- Noah A. Smith
- Yejin Choi
- Percy Liang
- Graham Neubig
- Yonatan Belinkov
- Jianfeng Gao
In Management Science and Operations Research
- Timo Schick
- Sebastian Gehrmann
- Yada Pruksachatkun
- Stella Biderman
- Jungo Kasai
- Teven Le Scao
- Irene Solaiman
- Rui Zhang
In Evaluation & Benchmarks
- Sebastian Gehrmann
- Yonatan Belinkov
- Verena Rieser
- Douwe Kiela
- Noah A. Smith
- Jekaterina Novikova
- Elizabeth Clark
- Ehud Reiter
Citations in the corpus
- BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, background): “, 2021), and existing carbon footprint studies have likely under-estimated emissions (Bannour et al., 2021).”
- Measuring the Carbon Intensity of AI in Cloud Instances (2022, background): “However, recent work has found that their results vary significantly and are not fully representative of the true emissions incurred by training [3].”
- Power Hungry Processing: Watts Driving the Cost of AI Deployment? (2023, methodology): “There are several tools that exist, such as Code Carbon [47], MLCO2 [26] and LLM-Carbon [13], all of which adopt different approaches and output different results (see [1] for a detailed comparison).”