CAFE: Carbon-Aware Federated Learning in Geographically Distributed Data Centers

• URL: http://arxiv.org/abs/2311.03615v2
• Date: Tue, 6 Feb 2024 02:56:12 GMT
• Title: CAFE: Carbon-Aware Federated Learning in Geographically Distributed Data Centers
• Authors: Jieming Bian, Lei Wang, Shaolei Ren, Jie Xu
• Abstract summary: Training large-scale artificial intelligence (AI) models demands significant computational power and energy, leading to increased carbon footprint with potential environmental repercussions. This paper delves into the challenges of training AI models across geographically distributed (geo-distributed) data centers, emphasizing the balance between learning performance and carbon footprint. We propose a new framework called CAFE (short for Carbon-Aware Federated Learning) to optimize training within a fixed carbon footprint budget.
• Score: 18.54380015603228
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training large-scale artificial intelligence (AI) models demands significant computational power and energy, leading to increased carbon footprint with potential environmental repercussions. This paper delves into the challenges of training AI models across geographically distributed (geo-distributed) data centers, emphasizing the balance between learning performance and carbon footprint. We consider Federated Learning (FL) as a solution, which prioritizes model parameter exchange over raw data, ensuring data privacy and compliance with local regulations. Given the variability in carbon intensity across regions, we propose a new framework called CAFE (short for Carbon-Aware Federated Learning) to optimize training within a fixed carbon footprint budget. Our approach incorporates coreset selection to assess learning performance, employs the Lyapunov drift-plus-penalty framework to address the unpredictability of future carbon intensity, and devises an efficient algorithm to address the combinatorial complexity of the data center selection. Through extensive simulations using real-world carbon intensity data, we demonstrate the efficacy of our algorithm, highlighting its superiority over existing methods in optimizing learning performance while minimizing environmental impact.

Related papers

• Center-Sensitive Kernel Optimization for Efficient On-Device Incremental Learning [88.78080749909665]
  Current on-device training methods just focus on efficient training without considering the catastrophic forgetting. This paper proposes a simple but effective edge-friendly incremental learning framework. Our method achieves average accuracy boost of 38.08% with even less memory and approximate computation.
  arXiv  Detail & Related papers  (2024-06-13T05:49:29Z)
• CarbonSense: A Multimodal Dataset and Baseline for Carbon Flux Modelling [9.05128569357374]
  We present CarbonSense, the first machine learning-ready dataset for data-driven carbon flux modelling. Our experiments illustrate the potential gains that multimodal deep learning techniques can bring to this domain.
  arXiv  Detail & Related papers  (2024-06-07T13:47:40Z)
• FedGreen: Carbon-aware Federated Learning with Model Size Adaptation [36.283273000969636]
  Federated learning (FL) provides a promising collaborative framework to build a model from distributed clients. Cloud and edge servers hosting FL clients may exhibit diverse carbon footprints influenced by their geographical locations with varying power sources. We propose FedGreen, a carbon-aware FL approach to efficiently train models by adopting adaptive model sizes shared with clients.
  arXiv  Detail & Related papers  (2024-04-23T20:37:26Z)
• A Carbon Tracking Model for Federated Learning: Impact of Quantization and Sparsification [5.341266334051207]
  Federated Learning (FL) methods adopt efficient communication technologies to distribute machine learning tasks across edge devices. This paper proposes a framework for real-time monitoring of the energy and carbon footprint impacts of FL systems.
  arXiv  Detail & Related papers  (2023-10-12T07:20:03Z)
• A Comparative Study of Machine Learning Algorithms for Anomaly Detection in Industrial Environments: Performance and Environmental Impact [62.997667081978825]
  This study seeks to address the demands of high-performance machine learning models with environmental sustainability. Traditional machine learning algorithms, such as Decision Trees and Random Forests, demonstrate robust efficiency and performance. However, superior outcomes were obtained with optimised configurations, albeit with a commensurate increase in resource consumption.
  arXiv  Detail & Related papers  (2023-07-01T15:18:00Z)
• Sustainable AIGC Workload Scheduling of Geo-Distributed Data Centers: A Multi-Agent Reinforcement Learning Approach [48.18355658448509]
  Recent breakthroughs in generative artificial intelligence have triggered a surge in demand for machine learning training, which poses significant cost burdens and environmental challenges due to its substantial energy consumption. Scheduling training jobs among geographically distributed cloud data centers unveils the opportunity to optimize the usage of computing capacity powered by inexpensive and low-carbon energy. We propose an algorithm based on multi-agent reinforcement learning and actor-critic methods to learn the optimal collaborative scheduling strategy through interacting with a cloud system built with real-life workload patterns, energy prices, and carbon intensities.
  arXiv  Detail & Related papers  (2023-04-17T02:12:30Z)
• Green Federated Learning [7.003870178055125]
  Federated Learning (FL) is a machine learning technique for training a centralized model using data of decentralized entities. FL may leverage as many as hundreds of millions of globally distributed end-user devices with diverse energy sources. We propose the concept of Green FL, which involves optimizing FL parameters and making design choices to minimize carbon emissions.
  arXiv  Detail & Related papers  (2023-03-26T02:23:38Z)
• Measuring the Carbon Intensity of AI in Cloud Instances [91.28501520271972]
  We provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions. We evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform.
  arXiv  Detail & Related papers  (2022-06-10T17:04:04Z)
• Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data [77.88594632644347]
  Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization challenge. We propose a novel momentum-based method to mitigate this decentralized training difficulty.
  arXiv  Detail & Related papers  (2021-02-09T11:27:14Z)
• Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning [68.37641996188133]
  We introduce a framework for tracking realtime energy consumption and carbon emissions. We create a leaderboard for energy efficient reinforcement learning algorithms. We propose strategies for mitigation of carbon emissions and reduction of energy consumption.
  arXiv  Detail & Related papers  (2020-01-31T05:12:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.