Realistic Endoscopic Image Generation Method Using Virtual-to-real Image-domain Translation

• URL: http://arxiv.org/abs/2201.04918v1
• Date: Thu, 13 Jan 2022 12:18:51 GMT
• Title: Realistic Endoscopic Image Generation Method Using Virtual-to-real Image-domain Translation
• Authors: Masahiro Oda, Kiyohito Tanaka, Hirotsugu Takabatake, Masaki Mori, Hiroshi Natori, Kensaku Mori
• Abstract summary: We propose a realistic image generation method for endoscopic simulation systems. Virtual endoscopic images are generated by using a volume rendering method from a CT volume of a patient. We improve the reality of the virtual endoscopic images using a virtual-to-real image-domain translation technique.
• Score: 1.1580916951856253
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a realistic image generation method for visualization in endoscopic simulation systems. Endoscopic diagnosis and treatment are performed in many hospitals. To reduce complications related to endoscope insertions, endoscopic simulation systems are used for training or rehearsal of endoscope insertions. However, current simulation systems generate non-realistic virtual endoscopic images. To improve the value of the simulation systems, improvement of reality of their generated images is necessary. We propose a realistic image generation method for endoscopic simulation systems. Virtual endoscopic images are generated by using a volume rendering method from a CT volume of a patient. We improve the reality of the virtual endoscopic images using a virtual-to-real image-domain translation technique. The image-domain translator is implemented as a fully convolutional network (FCN). We train the FCN by minimizing a cycle consistency loss function. The FCN is trained using unpaired virtual and real endoscopic images. To obtain high quality image-domain translation results, we perform an image cleansing to the real endoscopic image set. We tested to use the shallow U-Net, U-Net, deep U-Net, and U-Net having residual units as the image-domain translator. The deep U-Net and U-Net having residual units generated quite realistic images.

Related papers

• SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Models [1.6189876649941652]
  We introduce emphSurgicaL-CD, a consistency-distilled diffusion method to generate realistic surgical images. Our results demonstrate that our method outperforms GANs and diffusion-based approaches.
  arXiv  Detail & Related papers  (2024-08-19T09:19:25Z)
• MeshBrush: Painting the Anatomical Mesh with Neural Stylization for Endoscopy [0.8437187555622164]
  Style transfer is a promising approach to close the sim-to-real gap in medical endoscopy. rendering synthetic endoscopic videos by traversing pre-operative scans can generate structurally accurate simulations. CycleGAN can imitate realistic endoscopic images from these simulations, but they are unsuitable for video-to-video synthesis. We propose MeshBrush, a neural mesh stylization method to synthesize temporally consistent videos.
  arXiv  Detail & Related papers  (2024-04-03T18:40:48Z)
• EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting [53.38166294158047]
  EndoGSLAM is an efficient approach for endoscopic surgeries, which integrates streamlined representation and differentiable Gaussianization. Experiments show that EndoGSLAM achieves a better trade-off between intraoperative availability and reconstruction quality than traditional or neural SLAM approaches.
  arXiv  Detail & Related papers  (2024-03-22T11:27:43Z)
• Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training [51.16994853817024]
  This work focuses on designing an effective pre-training framework for 3D radiology images. We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-07-31T17:59:42Z)
• Translating Simulation Images to X-ray Images via Multi-Scale Semantic Matching [16.175115921436582]
  We propose a new method to translate simulation images from an endovascular simulator to X-ray images. We apply self-domain semantic matching to ensure that the input image and the generated image have the same positional semantic relationships. Our method generates realistic X-ray images and outperforms other state-of-the-art approaches by a large margin.
  arXiv  Detail & Related papers  (2023-04-16T04:49:46Z)
• Shape Reconstruction from Thoracoscopic Images using Self-supervised Virtual Learning [2.4493299476776778]
  Intraoperative shape reconstruction of organs from endoscopic camera images is a complex yet indispensable technique for image-guided surgery. We propose a framework for generative virtual learning of shape reconstruction using image translation with common latent variables between simulated and real images. In this study, we targeted the shape reconstruction of collapsed lungs from thoracoscopic images and confirmed that virtual learning could improve the similarity between real and simulated images.
  arXiv  Detail & Related papers  (2023-01-25T23:08:41Z)
• Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images [79.07468367923619]
  We develop convolutional neural generative coding (Conv-NGC) We implement a flexible neurobiologically-motivated algorithm that progressively refines latent state maps. We study the effectiveness of our brain-inspired neural system on the tasks of reconstruction and image denoising.
  arXiv  Detail & Related papers  (2022-11-22T06:42:41Z)
• A Shared Representation for Photorealistic Driving Simulators [83.5985178314263]
  We propose to improve the quality of generated images by rethinking the discriminator architecture. The focus is on the class of problems where images are generated given semantic inputs, such as scene segmentation maps or human body poses. We aim to learn a shared latent representation that encodes enough information to jointly do semantic segmentation, content reconstruction, along with a coarse-to-fine grained adversarial reasoning.
  arXiv  Detail & Related papers  (2021-12-09T18:59:21Z)
• Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation [60.663499381212425]
  We enhance the image quality by using a Wasserstein Generative Adversarial Network combined with a novel Adaptive Gradient Balancing technique. In MRI, our method minimizes artifacts, while maintaining a high-quality reconstruction that produces sharper images than other techniques.
  arXiv  Detail & Related papers  (2021-04-05T13:05:22Z)
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [67.9464567157846]
  We propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-06-29T12:53:58Z)
• A CNN-Based Blind Denoising Method for Endoscopic Images [19.373025463383385]
  Many low-quality endoscopic images exist due to limited illumination and complex environment in GI tract. This paper proposes a convolutional blind denoising network for endoscopic images.
  arXiv  Detail & Related papers  (2020-03-16T03:11:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.