CHOSEN: Contrastive Hypothesis Selection for Multi-View Depth Refinement

• URL: http://arxiv.org/abs/2404.02225v1
• Date: Tue, 2 Apr 2024 18:27:03 GMT
• Title: CHOSEN: Contrastive Hypothesis Selection for Multi-View Depth Refinement
• Authors: Di Qiu, Yinda Zhang, Thabo Beeler, Vladimir Tankovich, Christian Häne, Sean Fanello, Christoph Rhemann, Sergio Orts Escolano,
• Abstract summary: CHOSEN is a flexible, robust and effective multi-view depth refinement framework. It can be employed in any existing multi-view stereo pipeline.
• Score: 17.4479165692548
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose CHOSEN, a simple yet flexible, robust and effective multi-view depth refinement framework. It can be employed in any existing multi-view stereo pipeline, with straightforward generalization capability for different multi-view capture systems such as camera relative positioning and lenses. Given an initial depth estimation, CHOSEN iteratively re-samples and selects the best hypotheses, and automatically adapts to different metric or intrinsic scales determined by the capture system. The key to our approach is the application of contrastive learning in an appropriate solution space and a carefully designed hypothesis feature, based on which positive and negative hypotheses can be effectively distinguished. Integrated in a simple baseline multi-view stereo pipeline, CHOSEN delivers impressive quality in terms of depth and normal accuracy compared to many current deep learning based multi-view stereo pipelines.

Related papers

• Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving [22.58849429006898]
  Current multi-view depth estimation methods or single-view and multi-view fusion methods will fail when given noisy pose settings. We propose a single-view and multi-view fused depth estimation system, which adaptively integrates high-confident multi-view and single-view results. Our method outperforms state-of-the-art multi-view and fusion methods under robustness testing.
  arXiv  Detail & Related papers  (2024-03-12T11:18:35Z)
• Dive Deeper into Rectifying Homography for Stereo Camera Online Self-Calibration [18.089940434364234]
  We develop a novel online self-calibration algorithm for stereo cameras. We introduce four new evaluation metrics to quantify the robustness and accuracy of extrinsic parameter estimation. Our source code, demo video, and supplement are publicly available at mias.group/StereoCalibrator.
  arXiv  Detail & Related papers  (2023-09-19T04:52:13Z)
• Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessment [50.82681686110528]
  Blind Omnidirectional Image Quality Assessment (BOIQA) aims to objectively assess the human perceptual quality of omnidirectional images (ODIs) The quality assessment of ODIs is severely hampered by the fact that the existing BOIQA pipeline lacks the modeling of the observer's browsing process. We propose a novel multi-sequence network for BOIQA called Assessor360, which is derived from the realistic multi-assessor ODI quality assessment procedure.
  arXiv  Detail & Related papers  (2023-05-18T13:55:28Z)
• Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
  We introduce an approach that performs continuous modeling of per-pixel depth. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
  arXiv  Detail & Related papers  (2023-03-31T16:01:03Z)
• SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation [101.55622133406446]
  We propose a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras. Specifically, we employ a joint network to process all the surrounding views and propose a cross-view transformer to effectively fuse the information from multiple views. In experiments, our method achieves the state-of-the-art performance on the challenging multi-camera depth estimation datasets.
  arXiv  Detail & Related papers  (2022-04-07T17:58:47Z)
• Improving Monocular Visual Odometry Using Learned Depth [84.05081552443693]
  We propose a framework to exploit monocular depth estimation for improving visual odometry (VO) The core of our framework is a monocular depth estimation module with a strong generalization capability for diverse scenes. Compared with current learning-based VO methods, our method demonstrates a stronger generalization ability to diverse scenes.
  arXiv  Detail & Related papers  (2022-04-04T06:26:46Z)
• A Confidence-based Iterative Solver of Depths and Surface Normals for Deep Multi-view Stereo [41.527018997251744]
  We introduce a deep multi-view stereo (MVS) system that jointly predicts depths, surface normals and per-view confidence maps. The key to our approach is a novel solver that iteratively solves for per-view depth map and normal map. Our proposed solver consistently improves the depth quality over both conventional and deep learning based MVS pipelines.
  arXiv  Detail & Related papers  (2022-01-19T14:08:45Z)
• Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry [25.003116148843525]
  We propose MaGNet, a framework for fusing single-view depth probability with multi-view geometry. MaGNet achieves state-of-the-art performance on ScanNet, 7-Scenes and KITTI.
  arXiv  Detail & Related papers  (2021-12-15T14:56:53Z)
• MorphEyes: Variable Baseline Stereo For Quadrotor Navigation [13.830987813403018]
  We present a framework for quadrotor navigation based on a stereo camera system whose baseline can be adapted on-the-fly. We show that our variable baseline system is more accurate and robust in all three scenarios.
  arXiv  Detail & Related papers  (2020-11-05T20:04:35Z)
• Reversing the cycle: self-supervised deep stereo through enhanced monocular distillation [51.714092199995044]
  In many fields, self-supervised learning solutions are rapidly evolving and filling the gap with supervised approaches. We propose a novel self-supervised paradigm reversing the link between the two. In order to train deep stereo networks, we distill knowledge through a monocular completion network.
  arXiv  Detail & Related papers  (2020-08-17T07:40:22Z)
• OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems [88.41004332322788]
  We present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estimation. We integrate our omnidirectional depth estimates into the visual odometry (VO) and add a loop closing module for global consistency.
  arXiv  Detail & Related papers  (2020-03-18T05:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.