A first look into the carbon footprint of federated learning

By Xinchi Qiu, Titouan Parcollet, Daniel J. Beutel, Taner Topal, Akhil Mathur, Nicholas Donald Lane. 2020

122 citations recorded by the source corpus.

Despite impressive results, deep learning-based technologies also raise severe privacy and environmental concerns induced by the training procedure often conducted in data centers. In response, alternatives to centralized training such as Federated Learning (FL) have emerged. Perhaps unexpectedly, FL in particular is starting to be deployed at a global scale by companies that must adhere to new legal demands and policies originating from governments and the civil society for privacy protection. However, the potential environmental impact related to FL remains unclear and unexplored. This paper offers the first-ever systematic study of the carbon footprint of FL. First, we propose a rigorous model to quantify the carbon footprint, hence facilitating the investigation of the relationship between FL design and carbon emissions. Then, we compare the carbon footprint of FL to traditional centralized learning. We also formalize an early-stage FL optimization problem enabling the community to consider the importance of optimizing the rate of CO2 emissions jointly to the accuracy of neural networks. Finally, we highlight and connect the reported results to the future challenges and trends in FL to reduce its environmental impact, including algorithms efficiency, hardware capabilities, and stronger industry transparency.

Subfields: Artificial Intelligence, Computer Science Applications, Optimization & Training, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

Carole-Jean Wu

In Optimization & Training

In Scaling & Learning Theory

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, methodology): “We assumed a device power of 3W and a router power of 7.5W [115], [114].”
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout (2021, background): “…thus increasing the representation of low-tier devices (and by extension the correlated demographic groups), as well as adding to the overall compute capabilities of the distributed system as a whole, potentially offsetting part of the carbon footprint of centralised training data centers [56].”