RobustMVS: Single Domain Generalized Deep Multi-view Stereo

• URL: http://arxiv.org/abs/2405.09131v1
• Date: Wed, 15 May 2024 06:56:05 GMT
• Title: RobustMVS: Single Domain Generalized Deep Multi-view Stereo
• Authors: Hongbin Xu, Weitao Chen, Baigui Sun, Xuansong Xie, Wenxiong Kang,
• Abstract summary: This work focuses on the domain generalization problem in Multi-view Stereo (MVS) We build a novel MVS domain generalization benchmark including synthetic and real-world datasets. In contrast to conventional domain generalization benchmarks, we consider a more realistic but challenging scenario, where only one source domain is available for training.
• Score: 27.92012008096311
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the impressive performance of Multi-view Stereo (MVS) approaches given plenty of training samples, the performance degradation when generalizing to unseen domains has not been clearly explored yet. In this work, we focus on the domain generalization problem in MVS. To evaluate the generalization results, we build a novel MVS domain generalization benchmark including synthetic and real-world datasets. In contrast to conventional domain generalization benchmarks, we consider a more realistic but challenging scenario, where only one source domain is available for training. The MVS problem can be analogized back to the feature matching task, and maintaining robust feature consistency among views is an important factor for improving generalization performance. To address the domain generalization problem in MVS, we propose a novel MVS framework, namely RobustMVS. A DepthClustering-guided Whitening (DCW) loss is further introduced to preserve the feature consistency among different views, which decorrelates multi-view features from viewpoint-specific style information based on geometric priors from depth maps. The experimental results further show that our method achieves superior performance on the domain generalization benchmark.

Related papers

• Open-Vocabulary Domain Generalization in Urban-Scene Segmentation [83.15573353963235]
  Domain Generalization in Semantic Domain (DG-SS) aims to enable segmentation models to perform robustly in unseen environments.<n>Recent progress in Vision-Language Models (VLMs) has advanced Open-Vocabulary Semantic (OV-SS) by enabling models to recognize a broader range of concepts.<n>Yet, these models remain sensitive to domain shifts and struggle to maintain robustness when deployed in unseen environments.<n>We propose S2-Corr, a state-space-driven text-image correlation refinement mechanism that produces more consistent text-image correlations under distribution changes.
  arXiv  Detail & Related papers  (2026-02-21T14:32:27Z)
• Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation [59.58987161199141]
  Multi-domain applications face challenges like lack of diverse benchmarks and poor out-of-domain generalization. We introduce a diverse benchmark comprising a variety of question-answering tasks from 8 sources and covering 13 domains. Our findings highlight key strategies for improving multi-domain RAG robustness.
  arXiv  Detail & Related papers  (2025-04-03T09:03:40Z)
• Unified Domain Adaptive Semantic Segmentation [96.74199626935294]
  Unsupervised Adaptive Domain Semantic (UDA-SS) aims to transfer the supervision from a labeled source domain to an unlabeled target domain. We propose a Quad-directional Mixup (QuadMix) method, characterized by tackling distinct point attributes and feature inconsistencies. Our method outperforms the state-of-the-art works by large margins on four challenging UDA-SS benchmarks.
  arXiv  Detail & Related papers  (2023-11-22T09:18:49Z)
• Hierarchical Mutual Information Analysis: Towards Multi-view Clustering in The Wild [9.380271109354474]
  This work proposes a deep MVC framework where data recovery and alignment are fused in a hierarchically consistent way to maximize the mutual information among different views. To the best of our knowledge, this could be the first successful attempt to handle the missing and unaligned data problem separately with different learning paradigms.
  arXiv  Detail & Related papers  (2023-10-28T06:43:57Z)
• DealMVC: Dual Contrastive Calibration for Multi-view Clustering [78.54355167448614]
  We propose a novel Dual contrastive calibration network for Multi-View Clustering (DealMVC) We first design a fusion mechanism to obtain a global cross-view feature. Then, a global contrastive calibration loss is proposed by aligning the view feature similarity graph and the high-confidence pseudo-label graph. During the training procedure, the interacted cross-view feature is jointly optimized at both local and global levels.
  arXiv  Detail & Related papers  (2023-08-17T14:14:28Z)
• Domain-Expanded ASTE: Rethinking Generalization in Aspect Sentiment Triplet Extraction [67.54420015049732]
  Aspect Sentiment Triplet Extraction (ASTE) is a challenging task in sentiment analysis, aiming to provide fine-grained insights into human sentiments. Existing benchmarks are limited to two domains and do not evaluate model performance on unseen domains. We introduce a domain-expanded benchmark by annotating samples from diverse domains, enabling evaluation of models in both in-domain and out-of-domain settings.
  arXiv  Detail & Related papers  (2023-05-23T18:01:49Z)
• Randomized Adversarial Style Perturbations for Domain Generalization [49.888364462991234]
  We propose a novel domain generalization technique, referred to as Randomized Adversarial Style Perturbation (RASP) The proposed algorithm perturbs the style of a feature in an adversarial direction towards a randomly selected class, and makes the model learn against being misled by the unexpected styles observed in unseen target domains. We evaluate the proposed algorithm via extensive experiments on various benchmarks and show that our approach improves domain generalization performance, especially in large-scale benchmarks.
  arXiv  Detail & Related papers  (2023-04-04T17:07:06Z)
• TFS-ViT: Token-Level Feature Stylization for Domain Generalization [17.82872117103924]
  Vision Transformers (ViTs) have shown outstanding performance for a broad range of computer vision tasks. This paper presents a first Token-level Feature Stylization (TFS-ViT) approach for domain generalization. Our approach transforms token features by mixing the normalization statistics of images from different domains.
  arXiv  Detail & Related papers  (2023-03-28T03:00:28Z)
• Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization [113.03189252044773]
  We propose a unified framework, Style-HAllucinated Dual consistEncy learning (SHADE), to handle domain shift in various visual tasks. Our versatile SHADE can significantly enhance the generalization in various visual recognition tasks, including image classification, semantic segmentation and object detection.
  arXiv  Detail & Related papers  (2022-12-18T11:42:51Z)
• INDIGO: Intrinsic Multimodality for Domain Generalization [26.344372409315177]
  We study how multimodal information can be leveraged in an "intrinsic" way to make systems generalize under unseen domains. We propose IntriNsic multimodality for DomaIn GeneralizatiOn (INDIGO)
  arXiv  Detail & Related papers  (2022-06-13T05:41:09Z)
• Semi-Supervised Domain Generalization with Stochastic StyleMatch [90.98288822165482]
  In real-world applications, we might have only a few labels available from each source domain due to high annotation cost. In this work, we investigate semi-supervised domain generalization, a more realistic and practical setting. Our proposed approach, StyleMatch, is inspired by FixMatch, a state-of-the-art semi-supervised learning method based on pseudo-labeling.
  arXiv  Detail & Related papers  (2021-06-01T16:00:08Z)
• Domain Generalization Needs Stochastic Weight Averaging for Robustness on Domain Shifts [19.55308715031151]
  Domain generalization aims to learn a generalizable model to unseen target domains from multiple source domains. Recent benchmarks show that most approaches do not provide significant improvements compared to the simple empirical risk minimization. In this paper, we analyze how ERM works in views of domain-invariant feature learning and domain-specific normalization.
  arXiv  Detail & Related papers  (2021-02-17T06:42:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.