Toward Cross-Layer Energy Optimizations in AI Systems

• URL: http://arxiv.org/abs/2404.06675v2
• Date: Tue, 6 Aug 2024 03:33:41 GMT
• Title: Toward Cross-Layer Energy Optimizations in AI Systems
• Authors: Jae-Won Chung, Nishil Talati, Mosharaf Chowdhury,
• Abstract summary: Energy efficiency is likely to become the gating factor toward adoption of artificial intelligence. With the pervasive usage of artificial intelligence (AI) and machine learning (ML) tools and techniques, their energy efficiency is likely to become the gating factor toward adoption. This is because generative AI (GenAI) models are massive energy hogs. Inference consumes even more energy, because a model trained once serve millions.
• Score: 4.871463967255196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The "AI for Science, Energy, and Security" report from DOE outlines a significant focus on developing and optimizing artificial intelligence workflows for a foundational impact on a broad range of DOE missions. With the pervasive usage of artificial intelligence (AI) and machine learning (ML) tools and techniques, their energy efficiency is likely to become the gating factor toward adoption. This is because generative AI (GenAI) models are massive energy hogs: for instance, training a 200-billion parameter large language model (LLM) at Amazon is estimated to have taken 11.9 GWh, which is enough to power more than a thousand average U.S. households for a year. Inference consumes even more energy, because a model trained once serve millions. Given this scale, high energy efficiency is key to addressing the power delivery problem of constructing and operating new supercomputers and datacenters specialized for AI workloads. In that regard, we outline software- and architecture-level research challenges and opportunities, setting the stage for creating cross-layer energy optimizations in AI systems.

Related papers

• Intelligent Mobile AI-Generated Content Services via Interactive Prompt Engineering and Dynamic Service Provisioning [55.641299901038316]
  AI-generated content can organize collaborative Mobile AIGC Service Providers (MASPs) at network edges to provide ubiquitous and customized content for resource-constrained users. Such a paradigm faces two significant challenges: 1) raw prompts often lead to poor generation quality due to users' lack of experience with specific AIGC models, and 2) static service provisioning fails to efficiently utilize computational and communication resources. We develop an interactive prompt engineering mechanism that leverages a Large Language Model (LLM) to generate customized prompt corpora and employs Inverse Reinforcement Learning (IRL) for policy imitation.
  arXiv  Detail & Related papers  (2025-02-17T03:05:20Z)
• Exploring the sustainable scaling of AI dilemma: A projective study of corporations' AI environmental impacts [0.0]
  We propose a methodology to estimate the environmental impact of a company's AI portfolio. Results confirm that large generative AI models consume up to 4600x more energy than traditional models. Mitigating the environmental impact of Generative AI by 2030 requires coordinated efforts across the AI value chain.
  arXiv  Detail & Related papers  (2025-01-24T08:58:49Z)
• The Unseen AI Disruptions for Power Grids: LLM-Induced Transients [0.5749787074942511]
  AI infrastructure features ultra-low inertia, sharp power surge and dip, and a significant peak-idle power ratio. These never-seen-before characteristics make AI a very unique load and pose threats to the power grid reliability and resilience. This paper examines the scale of AI power consumption, analyzes AI transient behaviour in various scenarios, develops high-level mathematical models to depict AI workload behaviour and discusses the multifaceted challenges and opportunities they potentially bring to existing power grids.
  arXiv  Detail & Related papers  (2024-09-09T05:22:01Z)
• The Energy Cost of Artificial Intelligence of Things Lifecycle [0.44739156031315913]
  We propose a new metric, the Energy Cost of AI lifecycle (eCAL) eCAL captures the energy consumption throughout the architectural components and lifecycle of an AI-powered wireless system. We show that the better a model and the more it is used, the more energy efficient an inference is.
  arXiv  Detail & Related papers  (2024-08-01T13:23:15Z)
• Present and Future of AI in Renewable Energy Domain : A Comprehensive Survey [0.0]
  Artificial intelligence (AI) has become a crucial instrument for streamlining processes in various industries. Nine AI-based strategies are identified here to assist Renewable Energy (RE) in contemporary power systems. This study also addressed three main topics: using AI technology for renewable power generation, utilizing AI for renewable energy forecasting, and optimizing energy systems.
  arXiv  Detail & Related papers  (2024-06-22T04:36:09Z)
• Green Edge AI: A Contemporary Survey [46.11332733210337]
  The transformative power of AI is derived from the utilization of deep neural networks (DNNs) Deep learning (DL) is increasingly being transitioned to wireless edge networks in proximity to end-user devices (EUDs) Despite its potential, edge AI faces substantial challenges, mostly due to the dichotomy between the resource limitations of wireless edge networks and the resource-intensive nature of DL.
  arXiv  Detail & Related papers  (2023-12-01T04:04:37Z)
• Power Hungry Processing: Watts Driving the Cost of AI Deployment? [74.19749699665216]
  generative, multi-purpose AI systems promise a unified approach to building machine learning (ML) models into technology. This ambition of generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions
  arXiv  Detail & Related papers  (2023-11-28T15:09:36Z)
• On the Opportunities of Green Computing: A Survey [80.21955522431168]
  Artificial Intelligence (AI) has achieved significant advancements in technology and research with the development over several decades. The needs for high computing power brings higher carbon emission and undermines research fairness. To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
  arXiv  Detail & Related papers  (2023-11-01T11:16:41Z)
• AI Maintenance: A Robustness Perspective [91.28724422822003]
  We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance. We propose an AI model inspection framework to detect and mitigate robustness risks. Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
  arXiv  Detail & Related papers  (2023-01-08T15:02:38Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.