ICG-MVSNet: Learning Intra-view and Cross-view Relationships for Guidance in Multi-View Stereo

• URL: http://arxiv.org/abs/2503.21525v1
• Date: Thu, 27 Mar 2025 14:13:31 GMT
• Title: ICG-MVSNet: Learning Intra-view and Cross-view Relationships for Guidance in Multi-View Stereo
• Authors: Yuxi Hu, Jun Zhang, Zhe Zhang, Rafael Weilharter, Yuchen Rao, Kuangyi Chen, Runze Yuan, Friedrich Fraundorfer,
• Abstract summary: Multi-view Stereo (MVS) aims to estimate depth and reconstruct 3D point clouds from a series of overlapping images.<n>Recent learning-based MVS frameworks overlook the geometric information embedded in features and correlations, leading to weak cost matching.<n>We propose ICG-MVSNet, which explicitly integrates intra-view and cross-view relationships for depth estimation.
• Score: 14.506324605370436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-view Stereo (MVS) aims to estimate depth and reconstruct 3D point clouds from a series of overlapping images. Recent learning-based MVS frameworks overlook the geometric information embedded in features and correlations, leading to weak cost matching. In this paper, we propose ICG-MVSNet, which explicitly integrates intra-view and cross-view relationships for depth estimation. Specifically, we develop an intra-view feature fusion module that leverages the feature coordinate correlations within a single image to enhance robust cost matching. Additionally, we introduce a lightweight cross-view aggregation module that efficiently utilizes the contextual information from volume correlations to guide regularization. Our method is evaluated on the DTU dataset and Tanks and Temples benchmark, consistently achieving competitive performance against state-of-the-art works, while requiring lower computational resources.

Related papers

• Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation [63.15257949821558]
  Referring Remote Sensing Image (RRSIS) is a new challenge that combines computer vision and natural language processing. Traditional Referring Image (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery. We introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS.
  arXiv  Detail & Related papers  (2023-12-19T08:14:14Z)
• Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention [100.81495948184649]
  We present Perceiver-VL, a vision-and-language framework that efficiently handles high-dimensional multimodal inputs such as long videos and text. Our framework scales with linear complexity, in contrast to the quadratic complexity of self-attention used in many state-of-the-art transformer-based models.
  arXiv  Detail & Related papers  (2022-11-21T18:22:39Z)
• RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo [20.470182157606818]
  "Learning-to-optimize" paradigm iteratively indexes a plane-sweeping cost volume and regresses the depth map via a convolutional Gated Recurrent Unit (GRU) We conduct extensive experiments on real-world MVS datasets and show that our method achieves state-of-the-art performance in terms of both within-dataset evaluation and cross-dataset generalization.
  arXiv  Detail & Related papers  (2022-05-28T03:32:56Z)
• Decoupled Side Information Fusion for Sequential Recommendation [6.515279047538104]
  We propose Decoupled Side Information Fusion for Sequential Recommendation (DIF-SR) It moves the side information from the input to the attention layer and decouples the attention calculation of various side information and item representation. Our proposed solution stably outperforms state-of-the-art SR models.
  arXiv  Detail & Related papers  (2022-04-23T10:53:36Z)
• Global-and-Local Collaborative Learning for Co-Salient Object Detection [162.62642867056385]
  The goal of co-salient object detection (CoSOD) is to discover salient objects that commonly appear in a query group containing two or more relevant images. We propose a global-and-local collaborative learning architecture, which includes a global correspondence modeling (GCM) and a local correspondence modeling (LCM) The proposed GLNet is evaluated on three prevailing CoSOD benchmark datasets, demonstrating that our model trained on a small dataset (about 3k images) still outperforms eleven state-of-the-art competitors trained on some large datasets (about 8k-200k images)
  arXiv  Detail & Related papers  (2022-04-19T14:32:41Z)
• Multi-View Stereo Network with attention thin volume [0.0]
  We propose an efficient multi-view stereo (MVS) network for infering depth value from multiple RGB images. We introduce the self-attention mechanism to fully aggregate the dominant information from input images. We also introduce the group-wise correlation to feature aggregation, which greatly reduces the memory and calculation burden.
  arXiv  Detail & Related papers  (2021-10-16T11:51:23Z)
• Deep Relational Metric Learning [84.95793654872399]
  This paper presents a deep relational metric learning framework for image clustering and retrieval. We learn an ensemble of features that characterizes an image from different aspects to model both interclass and intraclass distributions. Experiments on the widely-used CUB-200-2011, Cars196, and Stanford Online Products datasets demonstrate that our framework improves existing deep metric learning methods and achieves very competitive results.
  arXiv  Detail & Related papers  (2021-08-23T09:31:18Z)
• AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Network [8.127449025802436]
  We present a novel recurrent multi-view stereo network based on long short-term memory (LSTM) with adaptive aggregation, namely AA-RMVSNet. We firstly introduce an intra-view aggregation module to adaptively extract image features by using context-aware convolution and multi-scale aggregation. We propose an inter-view cost volume aggregation module for adaptive pixel-wise view aggregation, which is able to preserve better-matched pairs among all views.
  arXiv  Detail & Related papers  (2021-08-09T06:10:48Z)
• Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking [54.58791377183574]
  Our novel hybrid recurrent multi-view stereo net consists of two core modules: 1) a light DRENet (Dense Reception Expanded) module to extract dense feature maps of original size with multi-scale context information, 2) a HU-LSTM (Hybrid U-LSTM) to regularize 3D matching volume into predicted depth map. Our method exhibits competitive performance to the state-of-the-art method while dramatically reduces memory consumption, which costs only $19.4%$ of R-MVSNet memory consumption.
  arXiv  Detail & Related papers  (2020-07-21T14:59:59Z)
• Agglomerative Neural Networks for Multi-view Clustering [109.55325971050154]
  We propose the agglomerative analysis to approximate the optimal consensus view. We present Agglomerative Neural Network (ANN) based on Constrained Laplacian Rank to cluster multi-view data directly. Our evaluations against several state-of-the-art multi-view clustering approaches on four popular datasets show the promising view-consensus analysis ability of ANN.
  arXiv  Detail & Related papers  (2020-05-12T05:39:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.