Learning Monocular Depth from Focus with Event Focal Stack

• URL: http://arxiv.org/abs/2405.06944v1
• Date: Sat, 11 May 2024 07:54:49 GMT
• Title: Learning Monocular Depth from Focus with Event Focal Stack
• Authors: Chenxu Jiang, Mingyuan Lin, Chi Zhang, Zhenghai Wang, Lei Yu,
• Abstract summary: We propose the EDFF Network to estimate sparse depth from the Event Focal Stack. We use the event voxel grid to encode intensity change information and project event time surface into the depth domain. A Focal-Distance-guided Cross-Modal Attention Module is presented to fuse the information mentioned above.
• Score: 6.200121342586474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Depth from Focus estimates depth by determining the moment of maximum focus from multiple shots at different focal distances, i.e. the Focal Stack. However, the limited sampling rate of conventional optical cameras makes it difficult to obtain sufficient focus cues during the focal sweep. Inspired by biological vision, the event camera records intensity changes over time in extremely low latency, which provides more temporal information for focus time acquisition. In this study, we propose the EDFF Network to estimate sparse depth from the Event Focal Stack. Specifically, we utilize the event voxel grid to encode intensity change information and project event time surface into the depth domain to preserve per-pixel focal distance information. A Focal-Distance-guided Cross-Modal Attention Module is presented to fuse the information mentioned above. Additionally, we propose a Multi-level Depth Fusion Block designed to integrate results from each level of a UNet-like architecture and produce the final output. Extensive experiments validate that our method outperforms existing state-of-the-art approaches.

Related papers

• Self-supervised Event-based Monocular Depth Estimation using Cross-modal Consistency [18.288912105820167]
  We propose a self-supervised event-based monocular depth estimation framework named EMoDepth. EMoDepth constrains the training process using the cross-modal consistency from intensity frames that are aligned with events in the pixel coordinate. In inference, only events are used for monocular depth prediction.
  arXiv  Detail & Related papers  (2024-01-14T07:16:52Z)
• Towards Real-World Focus Stacking with Deep Learning [97.34754533628322]
  We introduce a new dataset consisting of 94 high-resolution bursts of raw images with focus bracketing. This dataset is used to train the first deep learning algorithm for focus stacking capable of handling bursts of sufficient length for real-world applications.
  arXiv  Detail & Related papers  (2023-11-29T17:49:33Z)
• Improving Fast Auto-Focus with Event Polarity [5.376511424333543]
  This paper presents a new high-speed and accurate event-based focusing algorithm. Experiments on the public event-based autofocus dataset (EAD) show the robustness of the model. precise focus with less than one depth of focus is achieved within 0.004 seconds on our self-built high-speed focusing platform.
  arXiv  Detail & Related papers  (2023-03-15T13:36:13Z)
• Uncertainty Guided Depth Fusion for Spike Camera [49.41822923588663]
  We propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse predictions of monocular and stereo depth estimation networks for spike camera. Our framework is motivated by the fact that stereo spike depth estimation achieves better results at close range. In order to demonstrate the advantage of spike depth estimation over traditional camera depth estimation, we contribute a spike-depth dataset named CitySpike20K.
  arXiv  Detail & Related papers  (2022-08-26T13:04:01Z)
• Deep Depth from Focal Stack with Defocus Model for Camera-Setting Invariance [19.460887007137607]
  We propose a learning-based depth from focus/defocus (DFF) which takes a focal stack as input for estimating scene depth. We show that our method is robust against a synthetic-to-real domain gap, and exhibits state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-02-26T04:21:08Z)
• Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments [56.306567220448684]
  We propose a new learning based end-to-end depth prediction network which takes noisy raw I-ToF signals as well as an RGB image. We show more than 40% RMSE improvement on the final depth map compared to the baseline approach.
  arXiv  Detail & Related papers  (2021-12-07T15:04:14Z)
• Deep Depth from Focus with Differential Focus Volume [17.505649653615123]
  We propose a convolutional neural network (CNN) to find the best-focused pixels in a focal stack and infer depth from the focus estimation. The key innovation of the network is the novel deep differential focus volume (DFV)
  arXiv  Detail & Related papers  (2021-12-03T04:49:51Z)
• MEFNet: Multi-scale Event Fusion Network for Motion Deblurring [62.60878284671317]
  Traditional frame-based cameras inevitably suffer from motion blur due to long exposure times. As a kind of bio-inspired camera, the event camera records the intensity changes in an asynchronous way with high temporal resolution. In this paper, we rethink the event-based image deblurring problem and unfold it into an end-to-end two-stage image restoration network.
  arXiv  Detail & Related papers  (2021-11-30T23:18:35Z)
• Dynamic Fusion Network For Light Field Depth Estimation [32.64928379844675]
  We propose a dynamically multi modal learning strategy which incorporates RGB data and the focal stack in our framework. The success of our method is demonstrated by achieving the state of the art performance on two datasets.
  arXiv  Detail & Related papers  (2021-04-13T06:45:11Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)
• Defocus Blur Detection via Depth Distillation [64.78779830554731]
  We introduce depth information into DBD for the first time. In detail, we learn the defocus blur from ground truth and the depth distilled from a well-trained depth estimation network. Our approach outperforms 11 other state-of-the-art methods on two popular datasets.
  arXiv  Detail & Related papers  (2020-07-16T04:58:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.