GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo

• URL: http://arxiv.org/abs/2404.07992v1
• Date: Thu, 11 Apr 2024 17:59:59 GMT
• Title: GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo
• Authors: Jiang Wu, Rui Li, Haofei Xu, Wenxun Zhao, Yu Zhu, Jinqiu Sun, Yanning Zhang,
• Abstract summary: We propose GoMVS to aggregate geometrically consistent costs, yielding better utilization of adjacent geometries. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets.
• Score: 45.5438546016874
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Matching cost aggregation plays a fundamental role in learning-based multi-view stereo networks. However, directly aggregating adjacent costs can lead to suboptimal results due to local geometric inconsistency. Related methods either seek selective aggregation or improve aggregated depth in the 2D space, both are unable to handle geometric inconsistency in the cost volume effectively. In this paper, we propose GoMVS to aggregate geometrically consistent costs, yielding better utilization of adjacent geometries. More specifically, we correspond and propagate adjacent costs to the reference pixel by leveraging the local geometric smoothness in conjunction with surface normals. We achieve this by the geometric consistent propagation (GCP) module. It computes the correspondence from the adjacent depth hypothesis space to the reference depth space using surface normals, then uses the correspondence to propagate adjacent costs to the reference geometry, followed by a convolution for aggregation. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets. Notably, our method ranks 1st on the Tanks & Temple Advanced benchmark.

Related papers

• Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views [45.125032766506536]
  We propose Sparse2DGS, an MVS-d Gaussian Splatting pipeline for complete and accurate reconstruction. Our key insight is to incorporate the geometric-prioritized enhancement schemes, allowing for direct and robust geometric learning under ill-posed conditions. Sparse2DGS outperforms existing methods by notable margins while being $2times$ faster than the NeRF-based fine-tuning approach.
  arXiv  Detail & Related papers  (2025-04-29T02:47:02Z)
• 3D Geometric Shape Assembly via Efficient Point Cloud Matching [59.241448711254485]
  We introduce Proxy Match Transform (PMT), an approximate high-order feature transform layer that enables reliable matching between mating surfaces of parts. Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task. We evaluate the proposed PMTR on the large-scale 3D geometric shape assembly benchmark dataset of Breaking Bad.
  arXiv  Detail & Related papers  (2024-07-15T08:50:02Z)
• Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images [56.86175251327466]
  We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context. Our approach extracts geometric context that encodes the geometric variations present in the input image and correlates depth estimation with geometric constraints. Our method unifies depth and surface normal estimations within a cohesive framework, which enables the generation of high-quality 3D geometry from images.
  arXiv  Detail & Related papers  (2024-02-08T17:57:59Z)
• Depth Completion using Geometry-Aware Embedding [22.333381291860498]
  This paper proposes an efficient method to learn geometry-aware embedding. It encodes the local and global geometric structure information from 3D points, e.g., scene layout, object's sizes and shapes, to guide dense depth estimation.
  arXiv  Detail & Related papers  (2022-03-21T12:06:27Z)
• Self-supervised Geometric Perception [96.89966337518854]
  Self-supervised geometric perception is a framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels. We show that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.
  arXiv  Detail & Related papers  (2021-03-04T15:34:43Z)
• Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking [54.58791377183574]
  Our novel hybrid recurrent multi-view stereo net consists of two core modules: 1) a light DRENet (Dense Reception Expanded) module to extract dense feature maps of original size with multi-scale context information, 2) a HU-LSTM (Hybrid U-LSTM) to regularize 3D matching volume into predicted depth map. Our method exhibits competitive performance to the state-of-the-art method while dramatically reduces memory consumption, which costs only $19.4%$ of R-MVSNet memory consumption.
  arXiv  Detail & Related papers  (2020-07-21T14:59:59Z)
• DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes [28.571946680616765]
  We propose a family of deep hierarchical convolutional networks over 3D geometric data. The first type, geodesic convolutions, defines the kernel weights over mesh surfaces or graphs. The second type, Euclidean convolutions, is independent of any underlying mesh structure.
  arXiv  Detail & Related papers  (2020-04-02T13:52:00Z)
• PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling [103.09504572409449]
  We propose a novel deep neural network based method, called PUGeo-Net, to generate uniform dense point clouds. Thanks to its geometry-centric nature, PUGeo-Net works well for both CAD models with sharp features and scanned models with rich geometric details.
  arXiv  Detail & Related papers  (2020-02-24T14:13:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.