DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume

• URL: http://arxiv.org/abs/2308.07225v1
• Date: Mon, 14 Aug 2023 15:57:42 GMT
• Title: DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume
• Authors: Xingyu Miao, Yang Bai, Haoran Duan, Yawen Huang, Fan Wan, Xinxing Xu, Yang Long, Yefeng Zheng
• Abstract summary: We propose a novel dynamic cost volume that exploits residual optical flow to describe moving objects. The results demonstrate that our model outperforms previously published baselines for self-supervised monocular depth estimation.
• Score: 26.990400985745786
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised monocular depth estimation methods typically rely on the reprojection error to capture geometric relationships between successive frames in static environments. However, this assumption does not hold in dynamic objects in scenarios, leading to errors during the view synthesis stage, such as feature mismatch and occlusion, which can significantly reduce the accuracy of the generated depth maps. To address this problem, we propose a novel dynamic cost volume that exploits residual optical flow to describe moving objects, improving incorrectly occluded regions in static cost volumes used in previous work. Nevertheless, the dynamic cost volume inevitably generates extra occlusions and noise, thus we alleviate this by designing a fusion module that makes static and dynamic cost volumes compensate for each other. In other words, occlusion from the static volume is refined by the dynamic volume, and incorrect information from the dynamic volume is eliminated by the static volume. Furthermore, we propose a pyramid distillation loss to reduce photometric error inaccuracy at low resolutions and an adaptive photometric error loss to alleviate the flow direction of the large gradient in the occlusion regions. We conducted extensive experiments on the KITTI and Cityscapes datasets, and the results demonstrate that our model outperforms previously published baselines for self-supervised monocular depth estimation.

Related papers

• Triply Laplacian Scale Mixture Modeling for Seismic Data Noise Suppression [51.87076090814921]
  Sparsity-based tensor recovery methods have shown great potential in suppressing seismic data noise. We propose a novel triply Laplacian scale mixture (TLSM) approach for seismic data noise suppression.
  arXiv  Detail & Related papers  (2025-02-20T08:28:01Z)
• Learning Monocular Depth from Events via Egomotion Compensation [20.388521240421948]
  Event cameras are neuromorphically inspired sensors that sparsely and asynchronously report brightness changes. We propose an interpretable monocular depth estimation framework, where the likelihood of various depth hypotheses is explicitly determined by the effect of motion compensation. Our proposed framework outperforms cutting-edge methods by up to 10% in terms of the absolute relative error metric.
  arXiv  Detail & Related papers  (2024-12-26T05:41:18Z)
• ProDepth: Boosting Self-Supervised Multi-Frame Monocular Depth with Probabilistic Fusion [17.448021191744285]
  Multi-frame monocular depth estimation relies on the geometric consistency between successive frames under the assumption of a static scene. The presence of moving objects in dynamic scenes introduces inevitable inconsistencies, causing misaligned multi-frame feature matching and misleading self-supervision during training. We propose a novel framework called ProDepth, which effectively addresses the mismatch problem caused by dynamic objects using a probabilistic approach.
  arXiv  Detail & Related papers  (2024-07-12T14:37:49Z)
• SMORE: Simulataneous Map and Object REconstruction [66.66729715211642]
  We present a method for dynamic surface reconstruction of large-scale urban scenes from LiDAR. We take a holistic perspective and optimize a compositional model of a dynamic scene that decomposes the world into rigidly-moving objects and the background.
  arXiv  Detail & Related papers  (2024-06-19T23:53:31Z)
• FlowDepth: Decoupling Optical Flow for Self-Supervised Monocular Depth Estimation [8.78717459496649]
  We propose FlowDepth, where a Dynamic Motion Flow Module (DMFM) decouples the optical flow by a mechanism-based approach and warps the dynamic regions thus solving the mismatch problem. For the unfairness of photometric errors caused by high-freq and low-texture regions, we use Depth-Cue-Aware Blur (DCABlur) and Cost-Volume sparsity loss respectively at the input and the loss level to solve the problem.
  arXiv  Detail & Related papers  (2024-03-28T10:31:23Z)
• DDN-SLAM: Real-time Dense Dynamic Neural Implicit SLAM [5.267859554944985]
  We introduce DDN-SLAM, the first real-time dense dynamic neural implicit SLAM system integrating semantic features. Compared to existing neural implicit SLAM systems, the tracking results on dynamic datasets indicate an average 90% improvement in Average Trajectory Error (ATE) accuracy.
  arXiv  Detail & Related papers  (2024-01-03T05:42:17Z)
• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Robust Dynamic Radiance Fields [79.43526586134163]
  Dynamic radiance field reconstruction methods aim to model the time-varying structure and appearance of a dynamic scene. Existing methods, however, assume that accurate camera poses can be reliably estimated by Structure from Motion (SfM) algorithms. We address this robustness issue by jointly estimating the static and dynamic radiance fields along with the camera parameters.
  arXiv  Detail & Related papers  (2023-01-05T18:59:51Z)
• Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics [74.1720528573331]
  Unsupervised monocular depth and ego-motion estimation has drawn extensive research attention in recent years. We propose DynaDepth, a novel scale-aware framework that integrates information from vision and IMU motion dynamics. We validate the effectiveness of DynaDepth by conducting extensive experiments and simulations on the KITTI and Make3D datasets.
  arXiv  Detail & Related papers  (2022-07-11T07:50:22Z)
• Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data [73.8970871148949]
  High-fidelity reconstruction of fluids from sparse multiview RGB videos remains a formidable challenge. Existing solutions either assume knowledge of obstacles and lighting, or only focus on simple fluid scenes without obstacles or complex lighting. We present the first method to reconstruct dynamic fluid by leveraging the governing physics (ie, Navier -Stokes equations) in an end-to-end optimization.
  arXiv  Detail & Related papers  (2022-06-14T03:38:08Z)
• Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth [37.021579239596164]
  Existing dynamic-object-focused methods only partially solved the mismatch problem at the training loss level. We propose a novel multi-frame monocular depth prediction method to solve these problems at both the prediction and supervision loss levels. Our method, called DynamicDepth, is a new framework trained via a self-supervised cycle consistent learning scheme.
  arXiv  Detail & Related papers  (2022-03-29T01:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.