Visualizing Missing Surfaces In Colonoscopy Videos using Shared Latent Space Representations

• URL: http://arxiv.org/abs/2101.07280v1
• Date: Mon, 18 Jan 2021 19:00:51 GMT
• Title: Visualizing Missing Surfaces In Colonoscopy Videos using Shared Latent Space Representations
• Authors: Shawn Mathew, Saad Nadeem and Arie Kaufman
• Abstract summary: Optical colonoscopy (OC) has a high miss rate due to a number of factors. We present a framework to visualize the missed regions per-frame during the colonoscopy.
• Score: 6.187780920448871
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical colonoscopy (OC), the most prevalent colon cancer screening tool, has a high miss rate due to a number of factors, including the geometry of the colon (haustral fold and sharp bends occlusions), endoscopist inexperience or fatigue, endoscope field of view, etc. We present a framework to visualize the missed regions per-frame during the colonoscopy, and provides a workable clinical solution. Specifically, we make use of 3D reconstructed virtual colonoscopy (VC) data and the insight that VC and OC share the same underlying geometry but differ in color, texture and specular reflections, embedded in the OC domain. A lossy unpaired image-to-image translation model is introduced with enforced shared latent space for OC and VC. This shared latent space captures the geometric information while deferring the color, texture, and specular information creation to additional Gaussian noise input. This additional noise input can be utilized to generate one-to-many mappings from VC to OC and OC to OC.

Related papers

• C3VDv2 -- Colonoscopy 3D video dataset with enhanced realism [1.1774995069145182]
  This paper introduces C3VDv2, the second version (v2) of the high-definition Colonoscopy 3D Video dataset.<n>192 video sequences were captured by imaging 60 unique, high-fidelity silicone colon phantom segments.<n>Eight simulated screening colonoscopy videos acquired by a gastroenterologist are provided with ground truth poses.<n>The dataset includes 15 videos featuring colon deformations for qualitative assessment.
  arXiv  Detail & Related papers  (2025-06-30T17:29:06Z)
• CoCPF: Coordinate-based Continuous Projection Field for Ill-Posed Inverse Problem in Imaging [78.734927709231]
  Sparse-view computed tomography (SVCT) reconstruction aims to acquire CT images based on sparsely-sampled measurements. Due to ill-posedness, implicit neural representation (INR) techniques may leave considerable holes'' (i.e., unmodeled spaces) in their fields, leading to sub-optimal results. We propose the Coordinate-based Continuous Projection Field (CoCPF), which aims to build hole-free representation fields for SVCT reconstruction.
  arXiv  Detail & Related papers  (2024-06-21T08:38:30Z)
• High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces [18.948630080040576]
  We introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map. Our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface. This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions.
  arXiv  Detail & Related papers  (2024-04-20T18:06:26Z)
• Multi-View Vertebra Localization and Identification from CT Images [57.56509107412658]
  We propose a multi-view vertebra localization and identification from CT images. We convert the 3D problem into a 2D localization and identification task on different views. Our method can learn the multi-view global information naturally.
  arXiv  Detail & Related papers  (2023-07-24T14:43:07Z)
• CLTS-GAN: Color-Lighting-Texture-Specular Reflection Augmentation for Colonoscopy [5.298287413134345]
  CLTS-GAN is a new deep learning model that gives fine control over color, lighting, texture, and specular reflection for OC video frames. We show that adding colonoscopy-specific augmentations to the training data can improve state-of-the-art polyp detection/segmentation methods.
  arXiv  Detail & Related papers  (2022-06-29T23:51:16Z)
• Virtual Correspondence: Humans as a Cue for Extreme-View Geometry [104.09449367670318]
  We present a novel concept called virtual correspondences (VCs) VCs conform with epipolar geometry; unlike classic correspondences, VCs do not need to be co-visible across views. We show how VCs can be seamlessly integrated with classic bundle adjustment to recover camera poses across extreme views.
  arXiv  Detail & Related papers  (2022-06-16T17:59:42Z)
• SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection [76.01333073259677]
  We propose the use of Space-aware Memory Queues for In-painting and Detecting anomalies from radiography images (abbreviated as SQUID) We show that SQUID can taxonomize the ingrained anatomical structures into recurrent patterns; and in the inference, it can identify anomalies (unseen/modified patterns) in the image.
  arXiv  Detail & Related papers  (2021-11-26T13:47:34Z)
• Multi-Modal MRI Reconstruction with Spatial Alignment Network [51.74078260367654]
  In clinical practice, magnetic resonance imaging (MRI) with multiple contrasts is usually acquired in a single study. Recent researches demonstrate that, considering the redundancy between different contrasts or modalities, a target MRI modality under-sampled in the k-space can be better reconstructed with the helps from a fully-sampled sequence. In this paper, we integrate the spatial alignment network with reconstruction, to improve the quality of the reconstructed target modality.
  arXiv  Detail & Related papers  (2021-08-12T08:46:35Z)
• FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos [6.187780920448871]
  Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays.
  arXiv  Detail & Related papers  (2021-06-23T16:41:10Z)
• Colonoscopy Polyp Detection: Domain Adaptation From Medical Report Images to Real-time Videos [76.37907640271806]
  We propose an Image-video-joint polyp detection network (Ivy-Net) to address the domain gap between colonoscopy images from historical medical reports and real-time videos. Experiments on the collected dataset demonstrate that our Ivy-Net achieves the state-of-the-art result on colonoscopy video.
  arXiv  Detail & Related papers  (2020-12-31T10:33:09Z)
• SERV-CT: A disparity dataset from CT for validation of endoscopic 3D reconstruction [8.448866668577946]
  We present a stereo-endoscopic reconstruction validation dataset based on CT (SERV-CT) The SERV-CT dataset provides an easy to use stereoscopic validation for surgical applications with smooth reference disparities and depths with coverage over the majority of the endoscopic images.
  arXiv  Detail & Related papers  (2020-12-22T01:28:30Z)
• Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation [5.861206243996454]
  We present a deep learning framework, Extended and Directional CycleGAN, for lossy unpaired image-to-image translation between optical colonoscopy (OC) and virtual colonoscopy (VC) We show results on scale-consistent depth inference for phantom, textured VC and for real polyp and normal colon video sequences. We also present results for realistic pendunculated and flat polyp synthesis from bumps introduced in 3D VC models.
  arXiv  Detail & Related papers  (2020-03-27T15:34:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.