MonoPCC: Photometric-invariant Cycle Constraint for Monocular Depth Estimation of Endoscopic Images

• URL: http://arxiv.org/abs/2404.16571v3
• Date: Fri, 13 Sep 2024 13:40:41 GMT
• Title: MonoPCC: Photometric-invariant Cycle Constraint for Monocular Depth Estimation of Endoscopic Images
• Authors: Zhiwei Wang, Ying Zhou, Shiquan He, Ting Li, Fan Huang, Qiang Ding, Xinxia Feng, Mei Liu, Qiang Li,
• Abstract summary: Photometric constraint is indispensable for self-supervised monocular depth estimation. Built-in light causes significant brightness fluctuations, making photometric constraint unreliable. We propose MonoPCC to address the brightness inconsistency radically by reshaping the photometric constraint into a cycle form.
• Score: 20.439758719616048
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Photometric constraint is indispensable for self-supervised monocular depth estimation. It involves warping a source image onto a target view using estimated depth&pose, and then minimizing the difference between the warped and target images. However, the endoscopic built-in light causes significant brightness fluctuations, and thus makes the photometric constraint unreliable. Previous efforts only mitigate this relying on extra models to calibrate image brightness. In this paper, we propose MonoPCC to address the brightness inconsistency radically by reshaping the photometric constraint into a cycle form. Instead of only warping the source image, MonoPCC constructs a closed loop consisting of two opposite forward-backward warping paths: from target to source and then back to target. Thus, the target image finally receives an image cycle-warped from itself, which naturally makes the constraint invariant to brightness changes. Moreover, MonoPCC transplants the source image's phase-frequency into the intermediate warped image to avoid structure lost, and also stabilizes the training via an exponential moving average (EMA) strategy to avoid frequent changes in the forward warping. The comprehensive and extensive experimental results on four endoscopic datasets demonstrate that our proposed MonoPCC shows a great robustness to the brightness inconsistency, and exceeds other state-of-the-arts by reducing the absolute relative error by at least 7.27%, 9.38%, 9.90% and 3.17%, respectively.

Related papers

• Physically-Based Photometric Bundle Adjustment in Non-Lambertian Environments [59.96101889715997]
  Photometric bundle adjustment (PBA) is widely used in estimating the camera pose and 3D geometry by assuming a Lambertian world. The assumption of photometric consistency is often violated since the non-diffuse reflection is common in real-world environments. We propose a novel physically-based PBA method to solve this problem.
  arXiv  Detail & Related papers  (2024-09-18T10:22:07Z)
• Fearless Luminance Adaptation: A Macro-Micro-Hierarchical Transformer for Exposure Correction [65.5397271106534]
  A single neural network is difficult to handle all exposure problems. In particular, convolutions hinder the ability to restore faithful color or details on extremely over-/under- exposed regions. We propose a Macro-Micro-Hierarchical transformer, which consists of a macro attention to capture long-range dependencies, a micro attention to extract local features, and a hierarchical structure for coarse-to-fine correction.
  arXiv  Detail & Related papers  (2023-09-02T09:07:36Z)
• Multipass wide-field phase imager [0.8317595522478687]
  We report a versatile interference contrast imaging technique with a large field-ofview of several mm2. Sensitivity is increased through the use of a self-imaging non-resonant cavity. Phase profiles can be resolved individually for each round thanks to a specially designed single-photon camera.
  arXiv  Detail & Related papers  (2023-06-30T10:56:00Z)
• LUT-GCE: Lookup Table Global Curve Estimation for Fast Low-light Image Enhancement [62.17015413594777]
  We present an effective and efficient approach for low-light image enhancement, named LUT-GCE. We estimate a global curve for the entire image that allows corrections for both under- and over-exposure. Our approach outperforms the state of the art in terms of inference speed, especially on high-definition images (e.g., 1080p and 4k)
  arXiv  Detail & Related papers  (2023-06-12T12:53:06Z)
• Fluctuation-based deconvolution in fluorescence microscopy using plug-and-play denoisers [2.236663830879273]
  spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light. Several deconvolution and super-resolution techniques have been proposed to overcome this limitation.
  arXiv  Detail & Related papers  (2023-03-20T15:43:52Z)
• Frequency-Aware Self-Supervised Monocular Depth Estimation [41.97188738587212]
  We present two versatile methods to enhance self-supervised monocular depth estimation models. The high generalizability of our methods is achieved by solving the fundamental and ubiquitous problems in photometric loss function. We are the first to propose blurring images to improve depth estimators with an interpretable analysis.
  arXiv  Detail & Related papers  (2022-10-11T14:30:26Z)
• Fast Two-step Blind Optical Aberration Correction [15.555393702795076]
  We propose a two-step scheme to correct optical aberrations in a single raw or JPEG image. First, we estimate local Gaussian blur kernels for overlapping patches and sharpen them with a non-blind deblurring technique. Second, we remove the remaining lateral chromatic aberrations with a convolutional neural network.
  arXiv  Detail & Related papers  (2022-08-01T16:04:46Z)
• Degradation-agnostic Correspondence from Resolution-asymmetric Stereo [96.03964515969652]
  We study the problem of stereo matching from a pair of images with different resolutions, e.g., those acquired with a tele-wide camera system. We propose to impose the consistency between two views in a feature space instead of the image space, named feature-metric consistency. We find that, although a stereo matching network trained with the photometric loss is not optimal, its feature extractor can produce degradation-agnostic and matching-specific features.
  arXiv  Detail & Related papers  (2022-04-04T12:24:34Z)
• Invertible Network for Unpaired Low-light Image Enhancement [78.33382003460903]
  We propose to leverage the invertible network to enhance low-light image in forward process and degrade the normal-light one inversely with unpaired learning. In addition to the adversarial loss, we design various loss functions to ensure the stability of training and preserve more image details. We present a progressive self-guided enhancement process for low-light images and achieve favorable performance against the SOTAs.
  arXiv  Detail & Related papers  (2021-12-24T17:00:54Z)
• Recurrent Exposure Generation for Low-Light Face Detection [113.25331155337759]
  We propose a novel Recurrent Exposure Generation (REG) module and a Multi-Exposure Detection (MED) module. REG produces progressively and efficiently intermediate images corresponding to various exposure settings. Such pseudo-exposures are then fused by MED to detect faces across different lighting conditions.
  arXiv  Detail & Related papers  (2020-07-21T17:30:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.