From Transformers to Large Language Models: A systematic review of AI applications in the energy sector towards Agentic Digital Twins

• URL: http://arxiv.org/abs/2506.06359v1
• Date: Tue, 03 Jun 2025 10:02:07 GMT
• Title: From Transformers to Large Language Models: A systematic review of AI applications in the energy sector towards Agentic Digital Twins
• Abstract summary: We review the rapid expanding field of AI applications in the energy domain focusing on Transformers and Large Language Models.<n>We highlight practical implementations, innovations, and areas where the research frontier is rapidly expanding.<n>We introduce the concept of the Agentic Digital Twin, a next-generation model that integrates LLMs to bring autonomy, proactivity, and social interaction into digital twin-based energy management systems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence (AI) has long promised to improve energy management in smart grids by enhancing situational awareness and supporting more effective decision-making. While traditional machine learning has demonstrated notable results in forecasting and optimization, it often struggles with generalization, situational awareness, and heterogeneous data integration. Recent advances in foundation models such as Transformer architecture and Large Language Models (LLMs) have demonstrated improved capabilities in modelling complex temporal and contextual relationships, as well as in multi-modal data fusion which is essential for most AI applications in the energy sector. In this review we synthesize the rapid expanding field of AI applications in the energy domain focusing on Transformers and LLMs. We examine the architectural foundations, domain-specific adaptations and practical implementations of transformer models across various forecasting and grid management tasks. We then explore the emerging role of LLMs in the field: adaptation and fine tuning for the energy sector, the type of tasks they are suited for, and the new challenges they introduce. Along the way, we highlight practical implementations, innovations, and areas where the research frontier is rapidly expanding. These recent developments reviewed underscore a broader trend: Generative AI (GenAI) is beginning to augment decision-making not only in high-level planning but also in day-to-day operations, from forecasting and grid balancing to workforce training and asset onboarding. Building on these developments, we introduce the concept of the Agentic Digital Twin, a next-generation model that integrates LLMs to bring autonomy, proactivity, and social interaction into digital twin-based energy management systems.

Related papers

• A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design [71.32556809242666]
  The rapid expansion of large-scale AI models has led to significant performance breakthroughs across diverse domains.<n>It has also raised critical concerns regarding computational costs, energy consumption, and environmental sustainability.<n>This survey aims to offer a holistic roadmap toward sustainable, scalable, and socially responsible development of large models.
  arXiv  Detail & Related papers  (2026-07-10T04:02:22Z)
• Networking-Aware Energy Efficiency in Agentic AI Inference: A Survey [64.20429885919701]
  We propose an energy accounting framework identifying computational and communication costs across the Perception-Reasoning-Action cycle.<n>We establish a unified taxonomy spanning model simplification, computation control, input and attention optimization, and hardware-aware inference.<n>We identify open challenges of federated green learning, carbon-aware agency, 6th generation mobile communication (6G)-native Agentic AI, and self-sustaining systems.
  arXiv  Detail & Related papers  (2026-04-09T06:13:59Z)
• RE-LLM: Integrating Large Language Models into Renewable Energy Systems [0.7466390172678973]
  We propose the Renewable Energy Large Language Model (RE-LLM), a hybrid framework that integrates Large Language Models (LLMs) directly into the energy system modeling workflow.<n>RE-LLM combines three core elements: (i) optimization-based scenario exploration, (ii) machine learning surrogates that accelerate computationally intensive simulations, and (iii) LLM-powered natural language generation that translates complex results into clear, stakeholder-oriented explanations.<n>It enables interactive, multilingual, and accessible engagement with future energy pathways, ultimately bridging the final gap between data-driven analysis and actionable decision-making for sustainable transitions.
  arXiv  Detail & Related papers  (2025-12-01T08:10:39Z)
• A Survey on Agentic Multimodal Large Language Models [84.18778056010629]
  We present a comprehensive survey on Agentic Multimodal Large Language Models (Agentic MLLMs)<n>We explore the emerging paradigm of agentic MLLMs, delineating their conceptual foundations and distinguishing characteristics from conventional MLLM-based agents.<n>To further accelerate research in this area for the community, we compile open-source training frameworks, training and evaluation datasets for developing agentic MLLMs.
  arXiv  Detail & Related papers  (2025-10-13T04:07:01Z)
• Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challenges [87.02855999212817]
  Edge General Intelligence (EGI) represents a transformative evolution of edge computing, where distributed agents possess the capability to perceive, reason, and act autonomously.<n>World models act as proactive internal simulators that not only predict but also actively imagine future trajectories, reason under uncertainty, and plan multi-step actions with foresight.<n>This survey bridges the gap by offering a comprehensive analysis of how world models can empower agentic artificial intelligence (AI) systems at the edge.
  arXiv  Detail & Related papers  (2025-08-13T07:29:40Z)
• Energy-Efficient Deep Reinforcement Learning with Spiking Transformers [3.037387520023979]
  Spiking neural networks (SNNs) offer an energy-efficient alternative for machine learning.<n>New Spike-Transformer Reinforcement Learning (STRL) algorithm combines the energy efficiency of SNNs with the powerful decision-making capabilities of reinforcement learning.<n>SNN Transformer achieves significantly improved policy performance compared to conventional agent-based Transformers.
  arXiv  Detail & Related papers  (2025-05-20T15:52:43Z)
• Generative AI for Autonomous Driving: Frontiers and Opportunities [145.6465312554513]
  This survey delivers a comprehensive synthesis of the emerging role of GenAI across the autonomous driving stack.<n>We begin by distilling the principles and trade-offs of modern generative modeling, encompassing VAEs, GANs, Diffusion Models, and Large Language Models.<n>We categorize practical applications, such as synthetic data generalization, end-to-end driving strategies, high-fidelity digital twin systems, smart transportation networks, and cross-domain transfer to embodied AI.
  arXiv  Detail & Related papers  (2025-05-13T17:59:20Z)
• Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey [59.52058740470727]
  Edge-cloud collaborative computing (ECCC) has emerged as a pivotal paradigm for addressing the computational demands of modern intelligent applications.<n>Recent advancements in AI, particularly deep learning and large language models (LLMs), have dramatically enhanced the capabilities of these distributed systems.<n>This survey provides a structured tutorial on fundamental architectures, enabling technologies, and emerging applications.
  arXiv  Detail & Related papers  (2025-05-03T13:55:38Z)
• A Survey of World Models for Autonomous Driving [63.33363128964687]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>World models offer high-fidelity representations of the driving environment that integrate multi-sensor data, semantic cues, and temporal dynamics.<n>This paper systematically reviews recent advances in world models for autonomous driving.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)
• Transforming the Hybrid Cloud for Emerging AI Workloads [82.21522417363666]
  This white paper envisions transforming hybrid cloud systems to meet the growing complexity of AI workloads.<n>The proposed framework addresses critical challenges in energy efficiency, performance, and cost-effectiveness.<n>This joint initiative aims to establish hybrid clouds as secure, efficient, and sustainable platforms.
  arXiv  Detail & Related papers  (2024-11-20T11:57:43Z)
• Collaborative AI in Sentiment Analysis: System Architecture, Data Prediction and Deployment Strategies [3.3374611485861116]
  Large language model (LLM) based artificial intelligence technologies have been a game-changer, particularly in sentiment analysis. However, integrating diverse AI models for processing complex multimodal data and the associated high costs of feature extraction presents significant challenges. This study introduces a collaborative AI framework designed to efficiently distribute and resolve tasks across various AI systems.
  arXiv  Detail & Related papers  (2024-10-17T06:14:34Z)
• Optimal Power Grid Operations with Foundation Models [0.0]
  We propose the use of AI Foundation Models (FMs) and advances in Graph Neural Networks to efficiently exploit poorly available grid data for different downstream tasks. For capturing the grid's underlying physics, we believe that building a self-supervised model learning the power flow dynamics is a critical first step towards developing an FM for the power grid.
  arXiv  Detail & Related papers  (2024-09-03T09:06:13Z)
• Towards General Industrial Intelligence: A Survey of Continual Large Models in Industrial IoT [25.16997700703974]
  This survey explores the integration of IIoT with large models (LMs) and their potential applications in industrial environments.<n>We focus on four primary types of industrial LMs: language-based, vision-based, time-series, and multimodal models.<n>We analyze how IIoT provides abundant and diverse data resources, supporting the training and fine-tuning of LMs.
  arXiv  Detail & Related papers  (2024-09-02T12:35:59Z)
• Generative AI Agent for Next-Generation MIMO Design: Fundamentals, Challenges, and Vision [76.4345564864002]
  Next-generation multiple input multiple output (MIMO) is expected to be intelligent and scalable. We propose the concept of the generative AI agent, which is capable of generating tailored and specialized contents. We present two compelling case studies that demonstrate the effectiveness of leveraging the generative AI agent for performance analysis.
  arXiv  Detail & Related papers  (2024-04-13T02:39:36Z)
• From Generative AI to Generative Internet of Things: Fundamentals, Framework, and Outlooks [82.964958051535]
  Generative Artificial Intelligence (GAI) possesses the capabilities of generating realistic data and facilitating advanced decision-making. By integrating GAI into modern Internet of Things (IoT), Generative Internet of Things (GIoT) is emerging and holds immense potential to revolutionize various aspects of society.
  arXiv  Detail & Related papers  (2023-10-27T02:58:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.