Aberration-Aware Depth-from-Focus

• URL: http://arxiv.org/abs/2303.04654v2
• Date: Mon, 17 Jul 2023 10:04:35 GMT
• Title: Aberration-Aware Depth-from-Focus
• Authors: Xinge Yang, Qiang Fu, Mohammed Elhoseiny, Wolfgang Heidrich
• Abstract summary: We investigate the domain gap caused by off-axis aberrations that will affect the decision of the best-focused frame in a focal stack. We then explore bridging this domain gap through aberration-aware training (AAT) Our approach involves a lightweight network that models lens aberrations at different positions and focus distances, which is then integrated into the conventional network training pipeline.
• Score: 20.956132508261664
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Computer vision methods for depth estimation usually use simple camera models with idealized optics. For modern machine learning approaches, this creates an issue when attempting to train deep networks with simulated data, especially for focus-sensitive tasks like Depth-from-Focus. In this work, we investigate the domain gap caused by off-axis aberrations that will affect the decision of the best-focused frame in a focal stack. We then explore bridging this domain gap through aberration-aware training (AAT). Our approach involves a lightweight network that models lens aberrations at different positions and focus distances, which is then integrated into the conventional network training pipeline. We evaluate the generality of pretrained models on both synthetic and real-world data. Our experimental results demonstrate that the proposed AAT scheme can improve depth estimation accuracy without fine-tuning the model or modifying the network architecture.

Related papers

• Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
  We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors. Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors. To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
  arXiv  Detail & Related papers  (2024-04-05T17:55:33Z)
• Fully Self-Supervised Depth Estimation from Defocus Clue [79.63579768496159]
  We propose a self-supervised framework that estimates depth purely from a sparse focal stack. We show that our framework circumvents the needs for the depth and AIF image ground-truth, and receives superior predictions.
  arXiv  Detail & Related papers  (2023-03-19T19:59:48Z)
• End-to-end Learning for Joint Depth and Image Reconstruction from Diffracted Rotation [10.896567381206715]
  We propose a novel end-to-end learning approach for depth from diffracted rotation. Our approach requires a significantly less complex model and less training data, yet it is superior to existing methods in the task of monocular depth estimation.
  arXiv  Detail & Related papers  (2022-04-14T16:14:37Z)
• Deep Depth from Focus with Differential Focus Volume [17.505649653615123]
  We propose a convolutional neural network (CNN) to find the best-focused pixels in a focal stack and infer depth from the focus estimation. The key innovation of the network is the novel deep differential focus volume (DFV)
  arXiv  Detail & Related papers  (2021-12-03T04:49:51Z)
• NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo [97.07453889070574]
  We present a new multi-view depth estimation method that utilizes both conventional SfM reconstruction and learning-based priors. We show that our proposed framework significantly outperforms state-of-the-art methods on indoor scenes.
  arXiv  Detail & Related papers  (2021-09-02T17:54:31Z)
• Occlusion-aware Unsupervised Learning of Depth from 4-D Light Fields [50.435129905215284]
  We present an unsupervised learning-based depth estimation method for 4-D light field processing and analysis. Based on the basic knowledge of the unique geometry structure of light field data, we explore the angular coherence among subsets of the light field views to estimate depth maps. Our method can significantly shrink the performance gap between the previous unsupervised method and supervised ones, and produce depth maps with comparable accuracy to traditional methods with obviously reduced computational cost.
  arXiv  Detail & Related papers  (2021-06-06T06:19:50Z)
• Domain Adaptive Monocular Depth Estimation With Semantic Information [13.387521845596149]
  We propose an adversarial training model that leverages semantic information to narrow the domain gap. The proposed compact model achieves state-of-the-art performance comparable to complex latest models.
  arXiv  Detail & Related papers  (2021-04-12T18:50:41Z)
• Deep Autofocus for Synthetic Aperture Sonar [28.306713374371814]
  In this letter, we demonstrate the potential of machine learning, specifically deep learning, to address the autofocus problem. We formulate the problem as a self-supervised, phase error estimation task using a deep network we call Deep Autofocus. Our results demonstrate Deep Autofocus can produce imagery that is perceptually as good as benchmark iterative techniques but at a substantially lower computational cost.
  arXiv  Detail & Related papers  (2020-10-29T15:31:15Z)
• Calibrating Self-supervised Monocular Depth Estimation [77.77696851397539]
  In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal. We show that incorporating prior information about the camera configuration and the environment, we can remove the scale ambiguity and predict depth directly, still using the self-supervised formulation and not relying on any additional sensors.
  arXiv  Detail & Related papers  (2020-09-16T14:35:45Z)
• Focus on defocus: bridging the synthetic to real domain gap for depth estimation [9.023847175654602]
  We tackle the issue of closing the synthetic-real domain gap by using domain invariant defocus blur as direct supervision. We leverage defocus cues by using a permutation invariant convolutional neural network that encourages the network to learn from the differences between images with a different point of focus. We are able to train our model completely on synthetic data and directly apply it to a wide range of real-world images.
  arXiv  Detail & Related papers  (2020-05-19T17:52:37Z)
• Single Image Depth Estimation Trained via Depth from Defocus Cues [105.67073923825842]
  Estimating depth from a single RGB image is a fundamental task in computer vision. In this work, we rely, instead of different views, on depth from focus cues. We present results that are on par with supervised methods on KITTI and Make3D datasets and outperform unsupervised learning approaches.
  arXiv  Detail & Related papers  (2020-01-14T20:22:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.