Toward Cross-Layer Energy Optimizations in AI Systems

• URL: http://arxiv.org/abs/2404.06675v2
• Date: Tue, 6 Aug 2024 03:33:41 GMT
• Title: Toward Cross-Layer Energy Optimizations in AI Systems
• Authors: Jae-Won Chung, Nishil Talati, Mosharaf Chowdhury,
• Abstract summary: Energy efficiency is likely to become the gating factor toward adoption of artificial intelligence. With the pervasive usage of artificial intelligence (AI) and machine learning (ML) tools and techniques, their energy efficiency is likely to become the gating factor toward adoption. This is because generative AI (GenAI) models are massive energy hogs. Inference consumes even more energy, because a model trained once serve millions.
• Score: 4.871463967255196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The "AI for Science, Energy, and Security" report from DOE outlines a significant focus on developing and optimizing artificial intelligence workflows for a foundational impact on a broad range of DOE missions. With the pervasive usage of artificial intelligence (AI) and machine learning (ML) tools and techniques, their energy efficiency is likely to become the gating factor toward adoption. This is because generative AI (GenAI) models are massive energy hogs: for instance, training a 200-billion parameter large language model (LLM) at Amazon is estimated to have taken 11.9 GWh, which is enough to power more than a thousand average U.S. households for a year. Inference consumes even more energy, because a model trained once serve millions. Given this scale, high energy efficiency is key to addressing the power delivery problem of constructing and operating new supercomputers and datacenters specialized for AI workloads. In that regard, we outline software- and architecture-level research challenges and opportunities, setting the stage for creating cross-layer energy optimizations in AI systems.

Related papers

• The Unseen AI Disruptions for Power Grids: LLM-Induced Transients [0.5749787074942511]
  AI infrastructure features ultra-low inertia, sharp power surge and dip, and a significant peak-idle power ratio. These never-seen-before characteristics make AI a very unique load and pose threats to the power grid reliability and resilience. This paper examines the scale of AI power consumption, analyzes AI transient behaviour in various scenarios, develops high-level mathematical models to depict AI workload behaviour and discusses the multifaceted challenges and opportunities they potentially bring to existing power grids.
  arXiv  Detail & Related papers  (2024-09-09T05:22:01Z)
• Present and Future of AI in Renewable Energy Domain : A Comprehensive Survey [0.0]
  Artificial intelligence (AI) has become a crucial instrument for streamlining processes in various industries. Nine AI-based strategies are identified here to assist Renewable Energy (RE) in contemporary power systems. This study also addressed three main topics: using AI technology for renewable power generation, utilizing AI for renewable energy forecasting, and optimizing energy systems.
  arXiv  Detail & Related papers  (2024-06-22T04:36:09Z)
• Green Edge AI: A Contemporary Survey [46.11332733210337]
  The transformative power of AI is derived from the utilization of deep neural networks (DNNs) Deep learning (DL) is increasingly being transitioned to wireless edge networks in proximity to end-user devices (EUDs) Despite its potential, edge AI faces substantial challenges, mostly due to the dichotomy between the resource limitations of wireless edge networks and the resource-intensive nature of DL.
  arXiv  Detail & Related papers  (2023-12-01T04:04:37Z)
• Power Hungry Processing: Watts Driving the Cost of AI Deployment? [74.19749699665216]
  generative, multi-purpose AI systems promise a unified approach to building machine learning (ML) models into technology. This ambition of generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions
  arXiv  Detail & Related papers  (2023-11-28T15:09:36Z)
• On the Opportunities of Green Computing: A Survey [80.21955522431168]
  Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades. The needs for high computing power brings higher carbon emission and undermines research fairness. To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
  arXiv  Detail & Related papers  (2023-11-01T11:16:41Z)
• AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges [60.56413461109281]
  Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes. We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful. We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
  arXiv  Detail & Related papers  (2023-04-10T15:38:12Z)
• AI Maintenance: A Robustness Perspective [91.28724422822003]
  We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance. We propose an AI model inspection framework to detect and mitigate robustness risks. Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
  arXiv  Detail & Related papers  (2023-01-08T15:02:38Z)
• Trends in Energy Estimates for Computing in AI/Machine Learning Accelerators, Supercomputers, and Compute-Intensive Applications [3.2634122554914]
  We examine the computational energy requirements of different systems driven by the geometrical scaling law. We show that energy efficiency due to geometrical scaling is slowing down. At the application level, general-purpose AI-ML methods can be computationally energy intensive.
  arXiv  Detail & Related papers  (2022-10-12T16:14:33Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• The Powerful Use of AI in the Energy Sector: Intelligent Forecasting [7.747343962518897]
  This paper proposes a methodology to develop, deploy, and evaluate AI systems in the energy sector. The goal is to provide a high level of confidence to energy utility users.
  arXiv  Detail & Related papers  (2021-11-03T05:30:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.