CL-MVSNet: Unsupervised Multi-view Stereo with Dual-level Contrastive Learning

• URL: http://arxiv.org/abs/2503.08219v1
• Date: Tue, 11 Mar 2025 09:39:06 GMT
• Title: CL-MVSNet: Unsupervised Multi-view Stereo with Dual-level Contrastive Learning
• Authors: Kaiqiang Xiong, Rui Peng, Zhe Zhang, Tianxing Feng, Jianbo Jiao, Feng Gao, Ronggang Wang,
• Abstract summary: We propose a new dual-level contrastive learning approach, named CL-MVSNet.<n>Specifically, our model integrates two contrastive branches into an unsupervised MVS framework to construct additional supervisory signals.<n>Our approach achieves state-of-the-art performance among all end-to-end unsupervised MVS frameworks and outperforms its supervised counterpart by a considerable margin without fine-tuning.
• Score: 32.65909515998849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unsupervised Multi-View Stereo (MVS) methods have achieved promising progress recently. However, previous methods primarily depend on the photometric consistency assumption, which may suffer from two limitations: indistinguishable regions and view-dependent effects, e.g., low-textured areas and reflections. To address these issues, in this paper, we propose a new dual-level contrastive learning approach, named CL-MVSNet. Specifically, our model integrates two contrastive branches into an unsupervised MVS framework to construct additional supervisory signals. On the one hand, we present an image-level contrastive branch to guide the model to acquire more context awareness, thus leading to more complete depth estimation in indistinguishable regions. On the other hand, we exploit a scene-level contrastive branch to boost the representation ability, improving robustness to view-dependent effects. Moreover, to recover more accurate 3D geometry, we introduce an L0.5 photometric consistency loss, which encourages the model to focus more on accurate points while mitigating the gradient penalty of undesirable ones. Extensive experiments on DTU and Tanks&Temples benchmarks demonstrate that our approach achieves state-of-the-art performance among all end-to-end unsupervised MVS frameworks and outperforms its supervised counterpart by a considerable margin without fine-tuning.

Related papers

• A Black-Box Evaluation Framework for Semantic Robustness in Bird's Eye View Detection [24.737984789074094]
  We develop a robustness evaluation framework that adversarially optimises three common semantic perturbations to deceive BEV models.<n>To address the challenge posed by optimising the semantic perturbation, we design a smoothed, distance-based surrogate function to replace the mAP metric.<n>We provide a benchmark on the semantic robustness of ten recent BEV models.
  arXiv  Detail & Related papers  (2024-12-18T14:53:38Z)
• Low-Light Video Enhancement via Spatial-Temporal Consistent Illumination and Reflection Decomposition [68.6707284662443]
  Low-Light Video Enhancement (LLVE) seeks to restore dynamic and static scenes plagued by severe invisibility and noise. One critical aspect is formulating a consistency constraint specifically for temporal-spatial illumination and appearance enhanced versions. We present an innovative video Retinex-based decomposition strategy that operates without the need for explicit supervision.
  arXiv  Detail & Related papers  (2024-05-24T15:56:40Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion [35.21106030549071]
  Diffusion Probabilistic Models (DPMs) are dominant force in text-to-image generation tasks. We propose an alternative view of state-of-the-art DPMs as a way of inverting advanced Vision-Language Models (VLMs) By directly optimizing images with the supervision of discriminative VLMs, the proposed method can potentially achieve a better text-image alignment.
  arXiv  Detail & Related papers  (2024-02-26T05:08:40Z)
• RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation [50.35403070279804]
  3D occupancy prediction is an emerging task that aims to estimate the occupancy states and semantics of 3D scenes using multi-view images. We propose RadOcc, a Rendering assisted distillation paradigm for 3D Occupancy prediction.
  arXiv  Detail & Related papers  (2023-12-19T03:39:56Z)
• EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity [13.02735046166494]
  Self-supervised monocular scene flow estimation has received increasing attention for its simple and economical sensor setup. We propose a superior model named EMR-MSF by borrowing the advantages of network architecture design under the scope of supervised learning. On the KITTI scene flow benchmark, our approach improves the SF-all metric of the state-of-the-art self-supervised monocular method by 44%.
  arXiv  Detail & Related papers  (2023-09-04T00:30:06Z)
• Robust Single Image Dehazing Based on Consistent and Contrast-Assisted Reconstruction [95.5735805072852]
  We propose a novel density-variational learning framework to improve the robustness of the image dehzing model. Specifically, the dehazing network is optimized under the consistency-regularized framework. Our method significantly surpasses the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-29T08:11:04Z)
• PatchMVSNet: Patch-wise Unsupervised Multi-View Stereo for Weakly-Textured Surface Reconstruction [2.9896482273918434]
  This paper proposes robust loss functions leveraging constraints beneath multi-view images to alleviate matching ambiguity. Our strategy can be implemented with arbitrary depth estimation frameworks and can be trained with arbitrary large-scale MVS datasets. Our method reaches the performance of the state-of-the-art methods on popular benchmarks, like DTU, Tanks and Temples and ETH3D.
  arXiv  Detail & Related papers  (2022-03-04T07:05:23Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)
• Digging into Uncertainty in Self-supervised Multi-view Stereo [57.04768354383339]
  We propose a novel Uncertainty reduction Multi-view Stereo (UMVS) framework for self-supervised learning. Our framework achieves the best performance among unsupervised MVS methods, with competitive performance with its supervised opponents.
  arXiv  Detail & Related papers  (2021-08-30T02:53:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.