CL-MVSNet: Unsupervised Multi-view Stereo with Dual-level Contrastive Learning

• URL: http://arxiv.org/abs/2503.08219v1
• Date: Tue, 11 Mar 2025 09:39:06 GMT
• Title: CL-MVSNet: Unsupervised Multi-view Stereo with Dual-level Contrastive Learning
• Abstract summary: We propose a new dual-level contrastive learning approach, named CL-MVSNet.<n>Specifically, our model integrates two contrastive branches into an unsupervised MVS framework to construct additional supervisory signals.<n>Our approach achieves state-of-the-art performance among all end-to-end unsupervised MVS frameworks and outperforms its supervised counterpart by a considerable margin without fine-tuning.
• Score: 32.65909515998849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unsupervised Multi-View Stereo (MVS) methods have achieved promising progress recently. However, previous methods primarily depend on the photometric consistency assumption, which may suffer from two limitations: indistinguishable regions and view-dependent effects, e.g., low-textured areas and reflections. To address these issues, in this paper, we propose a new dual-level contrastive learning approach, named CL-MVSNet. Specifically, our model integrates two contrastive branches into an unsupervised MVS framework to construct additional supervisory signals. On the one hand, we present an image-level contrastive branch to guide the model to acquire more context awareness, thus leading to more complete depth estimation in indistinguishable regions. On the other hand, we exploit a scene-level contrastive branch to boost the representation ability, improving robustness to view-dependent effects. Moreover, to recover more accurate 3D geometry, we introduce an L0.5 photometric consistency loss, which encourages the model to focus more on accurate points while mitigating the gradient penalty of undesirable ones. Extensive experiments on DTU and Tanks&Temples benchmarks demonstrate that our approach achieves state-of-the-art performance among all end-to-end unsupervised MVS frameworks and outperforms its supervised counterpart by a considerable margin without fine-tuning.

Related papers

• Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model [73.59026525350808]
  We propose a dual adversarial fine-tuning framework that jointly optimize visual and semantic supervision signals from two modalities.<n>Our approach outperforms the state-of-the-art method in adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering tasks.
  arXiv  Detail & Related papers  (2026-07-21T10:49:29Z)
• HarmoGS: Robust 3D Gaussian Splatting in the Wild via Conflict-Aware Gradient Harmonization [82.42818558196615]
  In-the-wild 3D Gaussian Splatting remains challenging due to transient distractors and illumination-induced cross-view appearance inconsistencies.<n>We propose a conflict-aware 3DGS framework that addresses this problem from both image-space supervision and gradient-level optimization.<n>Our method achieves state-of-the-art rendering quality under complex transient distractors and cross-view inconsistencies.
  arXiv  Detail & Related papers  (2026-05-13T06:47:09Z)
• FS-I2P:A Hierarchical Focus-Sweep Registration Network with Dynamically Allocated Depth [23.96591843663943]
  Image-to-point cloud registration is often challenged by viewpoint changes, cross-modal discrepancies, and repetitive textures.<n>Recent detection-free methods alleviate this issue by leveraging multi-scale features and transformer-based interactions.<n>Inspired by human behavior, we propose a Focus--Sweep'' paradigm to enhance multi-level cross-modal feature association.
  arXiv  Detail & Related papers  (2026-05-08T11:33:18Z)
• CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Models [23.357627415320025]
  multimodal models that combine understanding and generation fail to leverage their own generative capacity on degraded inputs.<n>We present CLEAR, a framework that connects the two capabilities through three progressive steps.<n> Experiments show that CLEAR substantially improves robustness on degraded inputs while preserving clean-image performance.
  arXiv  Detail & Related papers  (2026-04-06T15:54:00Z)
• Why not Collaborative Filtering in Dual View? Bridging Sparse and Dense Models [17.01882282913444]
  Collaborative filtering remains the cornerstone of modern recommender systems.<n>We propose SaD (Sparse and Dense), a unified framework that integrates the semantic expressiveness of dense embeddings with the structural reliability of sparse interaction patterns.<n>We show that aligning these dual views yields a strictly superior global SNR.
  arXiv  Detail & Related papers  (2026-01-14T08:47:07Z)
• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings [75.0617088717528]
  MoCa is a framework for transforming pre-trained VLM backbones into effective bidirectional embedding models.<n>MoCa consistently improves performance across MMEB and ViDoRe-v2 benchmarks, achieving new state-of-the-art results.
  arXiv  Detail & Related papers  (2025-06-29T06:41:00Z)
• A Black-Box Evaluation Framework for Semantic Robustness in Bird's Eye View Detection [24.737984789074094]
  We develop a robustness evaluation framework that adversarially optimises three common semantic perturbations to deceive BEV models.<n>To address the challenge posed by optimising the semantic perturbation, we design a smoothed, distance-based surrogate function to replace the mAP metric.<n>We provide a benchmark on the semantic robustness of ten recent BEV models.
  arXiv  Detail & Related papers  (2024-12-18T14:53:38Z)
• Low-Light Video Enhancement via Spatial-Temporal Consistent Illumination and Reflection Decomposition [68.6707284662443]
  Low-Light Video Enhancement (LLVE) seeks to restore dynamic and static scenes plagued by severe invisibility and noise. One critical aspect is formulating a consistency constraint specifically for temporal-spatial illumination and appearance enhanced versions. We present an innovative video Retinex-based decomposition strategy that operates without the need for explicit supervision.
  arXiv  Detail & Related papers  (2024-05-24T15:56:40Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion [35.21106030549071]
  Diffusion Probabilistic Models (DPMs) are dominant force in text-to-image generation tasks. We propose an alternative view of state-of-the-art DPMs as a way of inverting advanced Vision-Language Models (VLMs) By directly optimizing images with the supervision of discriminative VLMs, the proposed method can potentially achieve a better text-image alignment.
  arXiv  Detail & Related papers  (2024-02-26T05:08:40Z)
• RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation [50.35403070279804]
  3D occupancy prediction is an emerging task that aims to estimate the occupancy states and semantics of 3D scenes using multi-view images. We propose RadOcc, a Rendering assisted distillation paradigm for 3D Occupancy prediction.
  arXiv  Detail & Related papers  (2023-12-19T03:39:56Z)
• EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity [13.02735046166494]
  Self-supervised monocular scene flow estimation has received increasing attention for its simple and economical sensor setup. We propose a superior model named EMR-MSF by borrowing the advantages of network architecture design under the scope of supervised learning. On the KITTI scene flow benchmark, our approach improves the SF-all metric of the state-of-the-art self-supervised monocular method by 44%.
  arXiv  Detail & Related papers  (2023-09-04T00:30:06Z)
• Robust Single Image Dehazing Based on Consistent and Contrast-Assisted Reconstruction [95.5735805072852]
  We propose a novel density-variational learning framework to improve the robustness of the image dehzing model. Specifically, the dehazing network is optimized under the consistency-regularized framework. Our method significantly surpasses the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-29T08:11:04Z)
• PatchMVSNet: Patch-wise Unsupervised Multi-View Stereo for Weakly-Textured Surface Reconstruction [2.9896482273918434]
  This paper proposes robust loss functions leveraging constraints beneath multi-view images to alleviate matching ambiguity. Our strategy can be implemented with arbitrary depth estimation frameworks and can be trained with arbitrary large-scale MVS datasets. Our method reaches the performance of the state-of-the-art methods on popular benchmarks, like DTU, Tanks and Temples and ETH3D.
  arXiv  Detail & Related papers  (2022-03-04T07:05:23Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)
• Digging into Uncertainty in Self-supervised Multi-view Stereo [57.04768354383339]
  We propose a novel Uncertainty reduction Multi-view Stereo (UMVS) framework for self-supervised learning. Our framework achieves the best performance among unsupervised MVS methods, with competitive performance with its supervised opponents.
  arXiv  Detail & Related papers  (2021-08-30T02:53:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.