Enhancing Bronchoscopy Depth Estimation through Synthetic-to-Real Domain Adaptation

• URL: http://arxiv.org/abs/2411.04404v1
• Date: Thu, 07 Nov 2024 03:48:35 GMT
• Title: Enhancing Bronchoscopy Depth Estimation through Synthetic-to-Real Domain Adaptation
• Authors: Qingyao Tian, Huai Liao, Xinyan Huang, Lujie Li, Hongbin Liu,
• Abstract summary: We propose a transfer learning framework that leverages synthetic data with depth labels for training and adapts domain knowledge for accurate depth estimation in real bronchoscope data. Our network demonstrates improved depth prediction on real footage using domain adaptation compared to training solely on synthetic data, validating our approach.
• Score: 2.795503750654676
• License:
• Abstract: Monocular depth estimation has shown promise in general imaging tasks, aiding in localization and 3D reconstruction. While effective in various domains, its application to bronchoscopic images is hindered by the lack of labeled data, challenging the use of supervised learning methods. In this work, we propose a transfer learning framework that leverages synthetic data with depth labels for training and adapts domain knowledge for accurate depth estimation in real bronchoscope data. Our network demonstrates improved depth prediction on real footage using domain adaptation compared to training solely on synthetic data, validating our approach.

Related papers

• Structure-preserving Image Translation for Depth Estimation in Colonoscopy Video [1.0485739694839669]
  We propose a pipeline of structure-preserving synthetic-to-real (sim2real) image translation. This allows us to generate large quantities of realistic-looking synthetic images for supervised depth estimation. We also propose a dataset of hand-picked sequences from clinical colonoscopies to improve the image translation process.
  arXiv  Detail & Related papers  (2024-08-19T17:02:16Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement [12.857137513211866]
  We propose an effective model training pipeline consisting of a training data synthesis and a gaze estimation model for unsupervised domain adaptation. The proposed data synthesis leverages the single-image 3D reconstruction to expand the range of the head poses from the source domain without requiring a 3D facial shape dataset. We propose a disentangling autoencoder network to separate gaze-related features and introduce background augmentation consistency loss to utilize the characteristics of the synthetic source domain.
  arXiv  Detail & Related papers  (2023-05-25T15:15:03Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• 3D-PL: Domain Adaptive Depth Estimation with 3D-aware Pseudo-Labeling [37.315964084413174]
  We develop a domain adaptation framework via generating reliable pseudo ground truths of depth from real data to provide direct supervisions. Specifically, we propose two mechanisms for pseudo-labeling: 1) 2D-based pseudo-labels via measuring the consistency of depth predictions when images are with the same content but different styles; 2) 3D-aware pseudo-labels via a point cloud completion network that learns to complete the depth values in the 3D space.
  arXiv  Detail & Related papers  (2022-09-19T17:54:17Z)
• Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation [111.89519571205778]
  In this work, we propose an alternative domain-adaptive approach to depth estimation. Our novel two-step structure first trains a depth estimation network with labeled synthetic images in a supervised manner. The results of our experiments show that the proposed method improves the network's performance on real images by a considerable margin.
  arXiv  Detail & Related papers  (2021-09-24T08:11:34Z)
• Occlusion-aware Unsupervised Learning of Depth from 4-D Light Fields [50.435129905215284]
  We present an unsupervised learning-based depth estimation method for 4-D light field processing and analysis. Based on the basic knowledge of the unique geometry structure of light field data, we explore the angular coherence among subsets of the light field views to estimate depth maps. Our method can significantly shrink the performance gap between the previous unsupervised method and supervised ones, and produce depth maps with comparable accuracy to traditional methods with obviously reduced computational cost.
  arXiv  Detail & Related papers  (2021-06-06T06:19:50Z)
• An Adaptive Framework for Learning Unsupervised Depth Completion [59.17364202590475]
  We present a method to infer a dense depth map from a color image and associated sparse depth measurements. We show that regularization and co-visibility are related via the fitness of the model to data and can be unified into a single framework.
  arXiv  Detail & Related papers  (2021-06-06T02:27:55Z)
• Learning a Domain-Agnostic Visual Representation for Autonomous Driving via Contrastive Loss [25.798361683744684]
  Domain-Agnostic Contrastive Learning (DACL) is a two-stage unsupervised domain adaptation framework with cyclic adversarial training and contrastive loss. Our proposed approach achieves better performance in the monocular depth estimation task compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-10T07:06:03Z)
• Unsupervised Metric Relocalization Using Transform Consistency Loss [66.19479868638925]
  Training networks to perform metric relocalization traditionally requires accurate image correspondences. We propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
  arXiv  Detail & Related papers  (2020-11-01T19:24:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.