From Transformers to Large Language Models: A systematic review of AI applications in the energy sector towards Agentic Digital Twins

• URL: http://arxiv.org/abs/2506.06359v1
• Date: Tue, 03 Jun 2025 10:02:07 GMT
• Title: From Transformers to Large Language Models: A systematic review of AI applications in the energy sector towards Agentic Digital Twins
• Authors: Gabriel Antonesi, Tudor Cioara, Ionut Anghel, Vasilis Michalakopoulos, Elissaios Sarmas, Liana Toderean,
• Abstract summary: We review the rapid expanding field of AI applications in the energy domain focusing on Transformers and Large Language Models.<n>We highlight practical implementations, innovations, and areas where the research frontier is rapidly expanding.<n>We introduce the concept of the Agentic Digital Twin, a next-generation model that integrates LLMs to bring autonomy, proactivity, and social interaction into digital twin-based energy management systems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence (AI) has long promised to improve energy management in smart grids by enhancing situational awareness and supporting more effective decision-making. While traditional machine learning has demonstrated notable results in forecasting and optimization, it often struggles with generalization, situational awareness, and heterogeneous data integration. Recent advances in foundation models such as Transformer architecture and Large Language Models (LLMs) have demonstrated improved capabilities in modelling complex temporal and contextual relationships, as well as in multi-modal data fusion which is essential for most AI applications in the energy sector. In this review we synthesize the rapid expanding field of AI applications in the energy domain focusing on Transformers and LLMs. We examine the architectural foundations, domain-specific adaptations and practical implementations of transformer models across various forecasting and grid management tasks. We then explore the emerging role of LLMs in the field: adaptation and fine tuning for the energy sector, the type of tasks they are suited for, and the new challenges they introduce. Along the way, we highlight practical implementations, innovations, and areas where the research frontier is rapidly expanding. These recent developments reviewed underscore a broader trend: Generative AI (GenAI) is beginning to augment decision-making not only in high-level planning but also in day-to-day operations, from forecasting and grid balancing to workforce training and asset onboarding. Building on these developments, we introduce the concept of the Agentic Digital Twin, a next-generation model that integrates LLMs to bring autonomy, proactivity, and social interaction into digital twin-based energy management systems.

Related papers

• Energy-Efficient Deep Reinforcement Learning with Spiking Transformers [3.037387520023979]
  Spiking neural networks (SNNs) offer an energy-efficient alternative for machine learning.<n>New Spike-Transformer Reinforcement Learning (STRL) algorithm combines the energy efficiency of SNNs with the powerful decision-making capabilities of reinforcement learning.<n>SNN Transformer achieves significantly improved policy performance compared to conventional agent-based Transformers.
  arXiv  Detail & Related papers  (2025-05-20T15:52:43Z)
• Generative AI for Autonomous Driving: Frontiers and Opportunities [145.6465312554513]
  This survey delivers a comprehensive synthesis of the emerging role of GenAI across the autonomous driving stack.<n>We begin by distilling the principles and trade-offs of modern generative modeling, encompassing VAEs, GANs, Diffusion Models, and Large Language Models.<n>We categorize practical applications, such as synthetic data generalization, end-to-end driving strategies, high-fidelity digital twin systems, smart transportation networks, and cross-domain transfer to embodied AI.
  arXiv  Detail & Related papers  (2025-05-13T17:59:20Z)
• Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey [59.52058740470727]
  Edge-cloud collaborative computing (ECCC) has emerged as a pivotal paradigm for addressing the computational demands of modern intelligent applications.<n>Recent advancements in AI, particularly deep learning and large language models (LLMs), have dramatically enhanced the capabilities of these distributed systems.<n>This survey provides a structured tutorial on fundamental architectures, enabling technologies, and emerging applications.
  arXiv  Detail & Related papers  (2025-05-03T13:55:38Z)
• A Survey of World Models for Autonomous Driving [63.33363128964687]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>World models offer high-fidelity representations of the driving environment that integrate multi-sensor data, semantic cues, and temporal dynamics.<n>This paper systematically reviews recent advances in world models for autonomous driving.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)
• Transforming the Hybrid Cloud for Emerging AI Workloads [82.21522417363666]
  This white paper envisions transforming hybrid cloud systems to meet the growing complexity of AI workloads.<n>The proposed framework addresses critical challenges in energy efficiency, performance, and cost-effectiveness.<n>This joint initiative aims to establish hybrid clouds as secure, efficient, and sustainable platforms.
  arXiv  Detail & Related papers  (2024-11-20T11:57:43Z)
• Collaborative AI in Sentiment Analysis: System Architecture, Data Prediction and Deployment Strategies [3.3374611485861116]
  Large language model (LLM) based artificial intelligence technologies have been a game-changer, particularly in sentiment analysis. However, integrating diverse AI models for processing complex multimodal data and the associated high costs of feature extraction presents significant challenges. This study introduces a collaborative AI framework designed to efficiently distribute and resolve tasks across various AI systems.
  arXiv  Detail & Related papers  (2024-10-17T06:14:34Z)
• Optimal Power Grid Operations with Foundation Models [0.0]
  We propose the use of AI Foundation Models (FMs) and advances in Graph Neural Networks to efficiently exploit poorly available grid data for different downstream tasks. For capturing the grid's underlying physics, we believe that building a self-supervised model learning the power flow dynamics is a critical first step towards developing an FM for the power grid.
  arXiv  Detail & Related papers  (2024-09-03T09:06:13Z)
• Towards General Industrial Intelligence: A Survey of Continual Large Models in Industrial IoT [25.16997700703974]
  This survey explores the integration of IIoT with large models (LMs) and their potential applications in industrial environments.<n>We focus on four primary types of industrial LMs: language-based, vision-based, time-series, and multimodal models.<n>We analyze how IIoT provides abundant and diverse data resources, supporting the training and fine-tuning of LMs.
  arXiv  Detail & Related papers  (2024-09-02T12:35:59Z)
• Generative AI Agent for Next-Generation MIMO Design: Fundamentals, Challenges, and Vision [76.4345564864002]
  Next-generation multiple input multiple output (MIMO) is expected to be intelligent and scalable. We propose the concept of the generative AI agent, which is capable of generating tailored and specialized contents. We present two compelling case studies that demonstrate the effectiveness of leveraging the generative AI agent for performance analysis.
  arXiv  Detail & Related papers  (2024-04-13T02:39:36Z)
• From Generative AI to Generative Internet of Things: Fundamentals, Framework, and Outlooks [82.964958051535]
  Generative Artificial Intelligence (GAI) possesses the capabilities of generating realistic data and facilitating advanced decision-making. By integrating GAI into modern Internet of Things (IoT), Generative Internet of Things (GIoT) is emerging and holds immense potential to revolutionize various aspects of society.
  arXiv  Detail & Related papers  (2023-10-27T02:58:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.