Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images

• URL: http://arxiv.org/abs/2107.04644v1
• Date: Fri, 9 Jul 2021 19:40:20 GMT
• Title: Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images
• Authors: Baoru Huang, Jianqing Zheng, Anh Nguyen, David Tuch, Kunal Vyas, Stamatia Giannarou, Daniel S. Elson
• Abstract summary: We propose SADepth, a new self-supervised depth estimation method based on Generative Adversarial Networks. It consists of an encoder-decoder generator and a discriminator to incorporate geometry constraints during training. Experiments on two public datasets show that SADepth outperforms recent state-of-the-art unsupervised methods by a large margin.
• Score: 13.996932179049978
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dense depth estimation and 3D reconstruction of a surgical scene are crucial steps in computer assisted surgery. Recent work has shown that depth estimation from a stereo images pair could be solved with convolutional neural networks. However, most recent depth estimation models were trained on datasets with per-pixel ground truth. Such data is especially rare for laparoscopic imaging, making it hard to apply supervised depth estimation to real surgical applications. To overcome this limitation, we propose SADepth, a new self-supervised depth estimation method based on Generative Adversarial Networks. It consists of an encoder-decoder generator and a discriminator to incorporate geometry constraints during training. Multi-scale outputs from the generator help to solve the local minima caused by the photometric reprojection loss, while the adversarial learning improves the framework generation quality. Extensive experiments on two public datasets show that SADepth outperforms recent state-of-the-art unsupervised methods by a large margin, and reduces the gap between supervised and unsupervised depth estimation in laparoscopic images.

Related papers

• Advancing Depth Anything Model for Unsupervised Monocular Depth Estimation in Endoscopy [3.1186464715409983]
  We introduce a novel fine-tuning strategy for the Depth Anything Model. We integrate it with an intrinsic-based unsupervised monocular depth estimation framework. Our results on the SCARED dataset show that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-09-12T03:04:43Z)
• ToDER: Towards Colonoscopy Depth Estimation and Reconstruction with Geometry Constraint Adaptation [67.22294293695255]
  We propose a novel reconstruction pipeline with a bi-directional adaptation architecture named ToDER to get precise depth estimations. Experimental results demonstrate that our approach can precisely predict depth maps in both realistic and synthetic colonoscopy videos.
  arXiv  Detail & Related papers  (2024-07-23T14:24:26Z)
• High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces [18.948630080040576]
  We introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map. Our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface. This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions.
  arXiv  Detail & Related papers  (2024-04-20T18:06:26Z)
• Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy [0.2995885872626565]
  We develop a novel multi-task learning (MTL) approach with a shared encoder and two decoders, namely a surface normal decoder and a depth estimator. We demonstrate an improvement of 14.17% on relative error and 10.4% on $delta_1$ accuracy over the most accurate baseline state-of-the-art BTS approach.
  arXiv  Detail & Related papers  (2023-11-30T16:13:17Z)
• AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimation [51.143540967290114]
  We propose a method that unlocks a wide range of previously-infeasible geometric augmentations for unsupervised depth computation and estimation. This is achieved by reversing, or undo''-ing, geometric transformations to the coordinates of the output depth, warping the depth map back to the original reference frame.
  arXiv  Detail & Related papers  (2023-10-15T05:15:45Z)
• Robot Localization and Mapping Final Report -- Sequential Adversarial Learning for Self-Supervised Deep Visual Odometry [2.512491726995032]
  Visual odometry (VO) and SLAM have been using multi-view geometry via local structure from motion for decades. Deep neural networks to extract high level features is ubiquitous in computer vision. The goal of this work is to tackle these limitations of past approaches and to develop a method that can provide better depths and pose estimates.
  arXiv  Detail & Related papers  (2023-09-08T06:24:17Z)
• EndoDepthL: Lightweight Endoscopic Monocular Depth Estimation with CNN-Transformer [0.0]
  We propose a novel lightweight solution named EndoDepthL that integrates CNN and Transformers to predict multi-scale depth maps. Our approach includes optimizing the network architecture, incorporating multi-scale dilated convolution, and a multi-channel attention mechanism. To better evaluate the performance of monocular depth estimation in endoscopic imaging, we propose a novel complexity evaluation metric.
  arXiv  Detail & Related papers  (2023-08-04T21:38:29Z)
• Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation [111.89519571205778]
  In this work, we propose an alternative domain-adaptive approach to depth estimation. Our novel two-step structure first trains a depth estimation network with labeled synthetic images in a supervised manner. The results of our experiments show that the proposed method improves the network's performance on real images by a considerable margin.
  arXiv  Detail & Related papers  (2021-09-24T08:11:34Z)
• Adaptive confidence thresholding for monocular depth estimation [83.06265443599521]
  We propose a new approach to leverage pseudo ground truth depth maps of stereo images generated from self-supervised stereo matching methods. The confidence map of the pseudo ground truth depth map is estimated to mitigate performance degeneration by inaccurate pseudo depth maps. Experimental results demonstrate superior performance to state-of-the-art monocular depth estimation methods.
  arXiv  Detail & Related papers  (2020-09-27T13:26:16Z)
• Calibrating Self-supervised Monocular Depth Estimation [77.77696851397539]
  In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal. We show that incorporating prior information about the camera configuration and the environment, we can remove the scale ambiguity and predict depth directly, still using the self-supervised formulation and not relying on any additional sensors.
  arXiv  Detail & Related papers  (2020-09-16T14:35:45Z)
• Occlusion-Aware Depth Estimation with Adaptive Normal Constraints [85.44842683936471]
  We present a new learning-based method for multi-frame depth estimation from a color video. Our method outperforms the state-of-the-art in terms of depth estimation accuracy.
  arXiv  Detail & Related papers  (2020-04-02T07:10:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.