Digital Twin-enabled Multi-generation Control Co-Design with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2510.10694v1
• Date: Sun, 12 Oct 2025 16:46:05 GMT
• Title: Digital Twin-enabled Multi-generation Control Co-Design with Deep Reinforcement Learning
• Authors: Ying-Kuan Tsai, Vispi Karkaria, Yi-Ping Chen, Wei Chen,
• Abstract summary: Control Co-Design (CCD) integrates physical and control system design to improve the performance of dynamic and autonomous systems.<n>This paper presents a DT-enabled CCD framework that integrates Deep Reinforcement Learning (DRL) to jointly optimize physical design and controller.
• Score: 8.45003507603975
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Control Co-Design (CCD) integrates physical and control system design to improve the performance of dynamic and autonomous systems. Despite advances in uncertainty-aware CCD methods, real-world uncertainties remain highly unpredictable. Multi-generation design addresses this challenge by considering the full lifecycle of a product: data collected from each generation informs the design of subsequent generations, enabling progressive improvements in robustness and efficiency. Digital Twin (DT) technology further strengthens this paradigm by creating virtual representations that evolve over the lifecycle through real-time sensing, model updating, and adaptive re-optimization. This paper presents a DT-enabled CCD framework that integrates Deep Reinforcement Learning (DRL) to jointly optimize physical design and controller. DRL accelerates real-time decision-making by allowing controllers to continuously learn from data and adapt to uncertain environments. Extending this approach, the framework employs a multi-generation paradigm, where each cycle of deployment, operation, and redesign uses collected data to refine DT models, improve uncertainty quantification through quantile regression, and inform next-generation designs of both physical components and controllers. The framework is demonstrated on an active suspension system, where DT-enabled learning from road conditions and driving behaviors yields smoother and more stable control trajectories. Results show that the method significantly enhances dynamic performance, robustness, and efficiency. Contributions of this work include: (1) extending CCD into a lifecycle-oriented multi-generation framework, (2) leveraging DTs for continuous model updating and informed design, and (3) employing DRL to accelerate adaptive real-time decision-making.

Related papers

• DDP-WM: Disentangled Dynamics Prediction for Efficient World Models [79.53092337527382]
  We introduce DDP-WM, a novel world model centered on the principle of Disentangled Dynamics Prediction.<n>DDP-WM realizes this decomposition through an architecture that integrates efficient historical processing with dynamic localization.<n>Experiments demonstrate that DDP-WM achieves significant efficiency and performance across diverse tasks.
  arXiv  Detail & Related papers  (2026-02-02T08:04:25Z)
• SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices [72.0937240883345]
  Recent advances in diffusion transformers (DiTs) have set new standards in image generation, yet remain impractical for on-device deployment.<n>We present an efficient DiT framework tailored for mobile and edge devices that achieves transformer-level generation quality under strict resource constraints.
  arXiv  Detail & Related papers  (2026-01-13T07:46:46Z)
• Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making [48.998030470623384]
  offline decision-making requires reliable behaviors from fixed datasets without further interaction.<n>We propose a compositional model-based diffusion framework consisting of: (i) a planner that generates diverse, task-aligned trajectories; (ii) a dynamics model that enforces consistency with the underlying system dynamics; and (iii) a ranker module that selects behaviors aligned with the task objectives.
  arXiv  Detail & Related papers  (2025-12-09T06:26:02Z)
• Digital Twin-based Control Co-Design of Full Vehicle Active Suspensions via Deep Reinforcement Learning [8.45003507603975]
  Digital twins (DTs) and deep reinforcement learning (DRL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle.<n>This work presents a DT-based control co-design (CCD) framework for full-vehicle active suspensions.<n>We jointly optimize physical suspension components and control policies under varying driver behaviors and environmental uncertainties.
  arXiv  Detail & Related papers  (2025-12-03T15:41:35Z)
• Adaptive Digital Twin of Sheet Metal Forming via Proper Orthogonal Decomposition-Based Koopman Operator with Model Predictive Control [14.54038067557375]
  Digital Twin (DT) technologies are transforming manufacturing by enabling real-time prediction, monitoring, and control of complex processes.<n>This study presents an adaptive DT framework that integrates proper Orthogonal Decomposition (POD) for physics-aware dimensionality reduction.<n>Online Recursive Least Squares (RLS) algorithm is introduced to update the operator coefficients in real time, enabling continuous adaptation of the DT model.
  arXiv  Detail & Related papers  (2025-11-13T23:24:17Z)
• Generative AI-Aided QoE Maximization for RIS-Assisted Digital Twin Interaction [6.54922175613871]
  We investigate a quality of experience (QoE)-aware resource allocation problem for reconfigurable intelligent surface (RIS)-assisted digital twin (DT) interaction with uncertain evolution.<n>Our goal is to maximize the sum of all mobile users' joint subjective and objective QoE in DT interactions across various DT scenes.<n>We propose a novel GAI-aided approach, called the prompt-guided decision transformer integrated with zero-forcing optimization (PG-ZFO)
  arXiv  Detail & Related papers  (2025-05-15T02:00:29Z)
• KDC-Diff: A Latent-Aware Diffusion Model with Knowledge Retention for Memory-Efficient Image Generation [2.0250638970950905]
  KDC-Diff is a novel and scalable generative framework designed to significantly reduce computational overhead while maintaining high performance.<n>Our model demonstrates strong performance across FID, CLIP, KID, and LPIPS metrics while achieving substantial reductions in parameter count, inference time, and FLOPs.
  arXiv  Detail & Related papers  (2025-05-11T14:40:51Z)
• DDD-GenDT: Dynamic Data-driven Generative Digital Twin Framework [0.43127334486935653]
  Digital twin (DT) technology enables real-time simulation, prediction, and optimization of physical systems.<n>This work introduces DDD-GenDT, a dynamic data-driven generative digital twin framework grounded in the Dynamic Data-Driven Application Systems paradigm.<n>By leveraging generative AI, DDD-GenDT reduces reliance on extensive historical datasets, enabling DT construction in data-scarce settings.
  arXiv  Detail & Related papers  (2024-12-28T01:13:30Z)
• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving.<n>Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner.<n>Experiments conducted on nuScenes and Bench2Drive datasets demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)
• Sustainable Diffusion-based Incentive Mechanism for Generative AI-driven Digital Twins in Industrial Cyber-Physical Systems [65.22300383287904]
  Industrial Cyber-Physical Systems (ICPSs) are an integral component of modern manufacturing and industries.<n>By digitizing data throughout product life cycles, Digital Twins (DTs) in ICPSs enable a shift from current industrial infrastructures to intelligent and adaptive infrastructures.<n>GenAI can drive the construction and update of DTs to improve predictive accuracy and prepare for diverse smart manufacturing.
  arXiv  Detail & Related papers  (2024-08-02T10:47:10Z)
• Constructing and Evaluating Digital Twins: An Intelligent Framework for DT Development [11.40908718824589]
  Development of Digital Twins (DTs) represents a transformative advance for simulating and optimizing complex systems in a controlled digital space. This paper introduces an intelligent framework for the construction and evaluation of DTs, specifically designed to enhance the accuracy and utility of DTs in testing algorithmic performance. We propose a novel construction methodology that integrates deep learning-based policy gradient techniques to dynamically tune the DT parameters, ensuring high fidelity in the digital replication of physical systems.
  arXiv  Detail & Related papers  (2024-06-19T01:45:18Z)
• TranDRL: A Transformer-Driven Deep Reinforcement Learning Enabled Prescriptive Maintenance Framework [58.474610046294856]
  Industrial systems demand reliable predictive maintenance strategies to enhance operational efficiency and reduce downtime. This paper introduces an integrated framework that leverages the capabilities of the Transformer model-based neural networks and deep reinforcement learning (DRL) algorithms to optimize system maintenance actions.
  arXiv  Detail & Related papers  (2023-09-29T02:27:54Z)
• Using Neural Networks by Modelling Semi-Active Shock Absorber [0.0]
  A permanently increasing number of on-board automotive control systems requires new approaches to their digital mapping. Various methods applying neural networks (NN) can be good candidates for relevant digital twin (DT) tools in automotive control system design. The paper presents an approach, which demonstrates how the regression tasks can be efficiently handled by the modeling of a semi-active shock absorber.
  arXiv  Detail & Related papers  (2022-07-19T09:21:21Z)
• Edge Continual Learning for Dynamic Digital Twins over Wireless Networks [68.65520952712914]
  Digital twins (DTs) constitute a critical link between the real-world and the metaverse. In this paper, a novel edge continual learning framework is proposed to accurately model the evolving affinity between a physical twin and its corresponding cyber twin. The proposed framework achieves a simultaneously accurate and synchronous CT model that is robust to catastrophic forgetting.
  arXiv  Detail & Related papers  (2022-04-10T23:25:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.