sim2real: Cardiac MR Image Simulation-to-Real Translation via Unsupervised GANs

• URL: http://arxiv.org/abs/2208.04874v1
• Date: Tue, 9 Aug 2022 16:06:06 GMT
• Title: sim2real: Cardiac MR Image Simulation-to-Real Translation via Unsupervised GANs
• Authors: Sina Amirrajab, Yasmina Al Khalil, Cristian Lorenz, Jurgen Weese, Josien Pluim, and Marcel Breeuwer
• Abstract summary: We provide image simulation on virtual XCAT subjects with varying anatomies. We propose sim2real translation network to improve image realism. Our usability experiments suggest that sim2real data exhibits a good potential to augment training data and boost the performance of a segmentation algorithm.
• Score: 0.4433315630787158
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: There has been considerable interest in the MR physics-based simulation of a database of virtual cardiac MR images for the development of deep-learning analysis networks. However, the employment of such a database is limited or shows suboptimal performance due to the realism gap, missing textures, and the simplified appearance of simulated images. In this work we 1) provide image simulation on virtual XCAT subjects with varying anatomies, and 2) propose sim2real translation network to improve image realism. Our usability experiments suggest that sim2real data exhibits a good potential to augment training data and boost the performance of a segmentation algorithm.

Related papers

• SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality [136.82569085134554]
  Foundation models like ChatGPT and Sora that are trained on a huge scale of data have made a revolutionary social impact. It is extremely challenging for sensors in many different fields to collect similar scales of natural images to train strong foundation models. This work presents a simple and effective framework SimMAT to study an open problem: the transferability from vision foundation models trained on natural RGB images to other image modalities of different physical properties.
  arXiv  Detail & Related papers  (2024-09-12T14:38:21Z)
• X-Fake: Juggling Utility Evaluation and Explanation of Simulated SAR Images [49.546627070454456]
  The distribution inconsistency between real and simulated data is the main obstacle that influences the utility of simulated SAR images. We propose a novel trustworthy utility evaluation framework with a counterfactual explanation for simulated SAR images for the first time, denoted as X-Fake. The proposed framework is validated on four simulated SAR image datasets obtained from electromagnetic models and generative artificial intelligence approaches.
  arXiv  Detail & Related papers  (2024-07-28T09:27:53Z)
• Near-realtime Facial Animation by Deep 3D Simulation Super-Resolution [7.14576106770047]
  We present a neural network-based simulation framework that can efficiently and realistically enhance a facial performance produced by a low-cost, realtime physics-based simulation. We use face animation as an exemplar of such a simulation domain, where creating this semantic congruence is achieved by simply dialing in the same muscle actuation controls and skeletal pose in the two simulators. Our proposed neural network super-resolution framework generalizes from this training set to unseen expressions, compensates for modeling discrepancies between the two simulations due to limited resolution or cost-cutting approximations in the real-time variant, and does not require any semantic descriptors or parameters to
  arXiv  Detail & Related papers  (2023-05-05T00:09:24Z)
• Translating Simulation Images to X-ray Images via Multi-Scale Semantic Matching [16.175115921436582]
  We propose a new method to translate simulation images from an endovascular simulator to X-ray images. We apply self-domain semantic matching to ensure that the input image and the generated image have the same positional semantic relationships. Our method generates realistic X-ray images and outperforms other state-of-the-art approaches by a large margin.
  arXiv  Detail & Related papers  (2023-04-16T04:49:46Z)
• AptSim2Real: Approximately-Paired Sim-to-Real Image Translation [8.208569626646035]
  Sim-to-real transfer modifies simulated images to better match real-world data. AptSim2Real exploits the fact that simulators can generate scenes loosely resembling real-world scenes in terms of lighting, environment, and composition. Our novel training strategy results in significant qualitative and quantitative improvements, with up to a 24% improvement in FID score.
  arXiv  Detail & Related papers  (2023-03-09T06:18:44Z)
• A Lightweight Machine Learning Pipeline for LiDAR-simulation [8.18203294574182]
  We propose a lightweight approach for more realistic LiDAR simulation. The central idea is to cast the simulation into an image-to-image translation problem. This strategy enables to skip the sensor-specific, expensive and complex LiDAR physics simulation.
  arXiv  Detail & Related papers  (2022-08-05T12:45:53Z)
• DSSIM: a structural similarity index for floating-point data [68.8204255655161]
  We propose an alternative to the popular SSIM that can be applied directly to the floating point data, which we refer to as the Data SSIM (DSSIM) While we demonstrate the usefulness of the DSSIM in the context of evaluating differences due to lossy compression on large volumes of simulation data, the DSSIM may prove useful for many other applications involving simulation or image data.
  arXiv  Detail & Related papers  (2022-02-05T19:18:33Z)
• Fed-Sim: Federated Simulation for Medical Imaging [131.56325440976207]
  We introduce a physics-driven generative approach that consists of two learnable neural modules. We show that our data synthesis framework improves the downstream segmentation performance on several datasets.
  arXiv  Detail & Related papers  (2020-09-01T19:17:46Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)
• ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images [0.0]
  Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. ML-SIM is thus robust to noise and irregularities in the illumination patterns of the raw SIM input frames.
  arXiv  Detail & Related papers  (2020-03-24T18:42:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.