Image-based Deep Learning for Smart Digital Twins: a Review

• URL: http://arxiv.org/abs/2401.02523v1
• Date: Thu, 4 Jan 2024 20:17:25 GMT
• Title: Image-based Deep Learning for Smart Digital Twins: a Review
• Authors: Md Ruman Islam, Mahadevan Subramaniam, Pei-Chi Huang (Department of Computer Science, University of Nebraska at Omaha, Omaha, NE, USA)
• Abstract summary: Smart Digital twins (SDTs) are being increasingly used to virtually replicate and predict the behaviors of complex physical systems. Deep learning (DL) models have significantly enhanced the capabilities of SDTs. This paper focuses on various approaches and associated challenges in developing image-based SDTs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Smart Digital twins (SDTs) are being increasingly used to virtually replicate and predict the behaviors of complex physical systems through continual data assimilation enabling the optimization of the performance of these systems by controlling the actions of systems. Recently, deep learning (DL) models have significantly enhanced the capabilities of SDTs, particularly for tasks such as predictive maintenance, anomaly detection, and optimization. In many domains, including medicine, engineering, and education, SDTs use image data (image-based SDTs) to observe and learn system behaviors and control their behaviors. This paper focuses on various approaches and associated challenges in developing image-based SDTs by continually assimilating image data from physical systems. The paper also discusses the challenges involved in designing and implementing DL models for SDTs, including data acquisition, processing, and interpretation. In addition, insights into the future directions and opportunities for developing new image-based DL approaches to develop robust SDTs are provided. This includes the potential for using generative models for data augmentation, developing multi-modal DL models, and exploring the integration of DL with other technologies, including 5G, edge computing, and IoT. In this paper, we describe the image-based SDTs, which enable broader adoption of the digital twin DT paradigms across a broad spectrum of areas and the development of new methods to improve the abilities of SDTs in replicating, predicting, and optimizing the behavior of complex systems.

Related papers

• Deep learning-based Visual Measurement Extraction within an Adaptive Digital Twin Framework from Limited Data Using Transfer Learning [0.0]
  Digital Twins technology is revolutionizing decision-making in scientific research by integrating models and simulations with real-time data. This research proposes a novel approach using Artificial Intelligence. Convolutional Neural Networks are employed to analyze structural behaviors in real-time by correlating Digital Image Correlation speckle pattern images with deformation fields.
  arXiv  Detail & Related papers  (2024-10-07T18:10:12Z)
• Constructing and Evaluating Digital Twins: An Intelligent Framework for DT Development [11.40908718824589]
  Development of Digital Twins (DTs) represents a transformative advance for simulating and optimizing complex systems in a controlled digital space. This paper introduces an intelligent framework for the construction and evaluation of DTs, specifically designed to enhance the accuracy and utility of DTs in testing algorithmic performance. We propose a novel construction methodology that integrates deep learning-based policy gradient techniques to dynamically tune the DT parameters, ensuring high fidelity in the digital replication of physical systems.
  arXiv  Detail & Related papers  (2024-06-19T01:45:18Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• IT3D: Improved Text-to-3D Generation with Explicit View Synthesis [71.68595192524843]
  This study presents a novel strategy that leverages explicitly synthesized multi-view images to address these issues. Our approach involves the utilization of image-to-image pipelines, empowered by LDMs, to generate posed high-quality images. For the incorporated discriminator, the synthesized multi-view images are considered real data, while the renderings of the optimized 3D models function as fake data.
  arXiv  Detail & Related papers  (2023-08-22T14:39:17Z)
• StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data [129.92449761766025]
  We propose a novel data collection methodology that synchronously synthesizes images and dialogues for visual instruction tuning. This approach harnesses the power of generative models, marrying the abilities of ChatGPT and text-to-image generative models. Our research includes comprehensive experiments conducted on various datasets.
  arXiv  Detail & Related papers  (2023-08-20T12:43:52Z)
• Enabling Spatial Digital Twins: Technologies, Challenges, and Future Research Directions [13.210510790794006]
  A Digital Twin (DT) is a virtual replica of a physical object or system, created to monitor, analyze, and optimize its behavior and characteristics. A Spatial Digital Twin (SDT) is a specific type of digital twin that emphasizes the geospatial aspects of the physical entity. We are the first to systematically analyze different spatial technologies relevant to building an SDT in layered approach.
  arXiv  Detail & Related papers  (2023-06-11T06:28:44Z)
• Systems for Parallel and Distributed Large-Model Deep Learning Training [7.106986689736828]
  Some recent Transformer models span hundreds of billions of learnable parameters. These designs have introduced new scale-driven systems challenges for the DL space. This survey will explore the large-model training systems landscape, highlighting key challenges and the various techniques that have been used to address them.
  arXiv  Detail & Related papers  (2023-01-06T19:17:29Z)
• Using Neural Networks by Modelling Semi-Active Shock Absorber [0.0]
  A permanently increasing number of on-board automotive control systems requires new approaches to their digital mapping. Various methods applying neural networks (NN) can be good candidates for relevant digital twin (DT) tools in automotive control system design. The paper presents an approach, which demonstrates how the regression tasks can be efficiently handled by the modeling of a semi-active shock absorber.
  arXiv  Detail & Related papers  (2022-07-19T09:21:21Z)
• Interpretable Hyperspectral AI: When Non-Convex Modeling meets Hyperspectral Remote Sensing [57.52865154829273]
  Hyperspectral imaging, also known as image spectrometry, is a landmark technique in geoscience remote sensing (RS) In the past decade efforts have been made to process analyze these hyperspectral (HS) products mainly by means of seasoned experts. For this reason, it is urgent to develop more intelligent and automatic approaches for various HS RS applications.
  arXiv  Detail & Related papers  (2021-03-02T03:32:10Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.