GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo

• URL: http://arxiv.org/abs/2404.07992v1
• Date: Thu, 11 Apr 2024 17:59:59 GMT
• Title: GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo
• Abstract summary: We propose GoMVS to aggregate geometrically consistent costs, yielding better utilization of adjacent geometries. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets.
• Score: 45.5438546016874
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Matching cost aggregation plays a fundamental role in learning-based multi-view stereo networks. However, directly aggregating adjacent costs can lead to suboptimal results due to local geometric inconsistency. Related methods either seek selective aggregation or improve aggregated depth in the 2D space, both are unable to handle geometric inconsistency in the cost volume effectively. In this paper, we propose GoMVS to aggregate geometrically consistent costs, yielding better utilization of adjacent geometries. More specifically, we correspond and propagate adjacent costs to the reference pixel by leveraging the local geometric smoothness in conjunction with surface normals. We achieve this by the geometric consistent propagation (GCP) module. It computes the correspondence from the adjacent depth hypothesis space to the reference depth space using surface normals, then uses the correspondence to propagate adjacent costs to the reference geometry, followed by a convolution for aggregation. Our method achieves new state-of-the-art performance on DTU, Tanks & Temple, and ETH3D datasets. Notably, our method ranks 1st on the Tanks & Temple Advanced benchmark.

Related papers

• Stable and Scalable Bundle Adjustment of Holistic 3D Structures [99.03818732955783]
  We present a new version of Bundle Adjustment (BA) for 3D computer vision.<n>BA is developed to jointly optimize cameras, poses and 3D points.<n>By group-induced measurements and cross-feature reprojection, we preserve the sparsity of classical point-only 3D regularization.
  arXiv  Detail & Related papers  (2026-09-03T16:08:03Z)
• Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models [12.061080782435013]
  Recent Vision Foundation Models (VFMs) predict depth, camera pose, and pointmap in a single forward pass without per-scene optimization.<n> enforcing explicit multi-view geometric consistency, e.g., through bundle adjustment, is computationally costly and is thus not imposed during VFM pretraining.<n>We propose Self-Geometry, a plug-and-play test-time adaptation pipeline that directly imposes explicit multi-view geometric constraints using 2D pixel correspondences as pseudo ground-truth.
  arXiv  Detail & Related papers  (2026-08-11T09:28:50Z)
• D^2-4DGS: Dual-Depth Guided Sparse-Camera 4D Gaussian Splatting [0.0]
  We propose a sparse-camera dynamic 4D Gaussian Splatting framework guided by dual-source depth priors.<n>Across all nine dataset--view settings, D$2$-4DGS achieves the highest PSNR, improving by 1.33 dB on average over the best competing method.
  arXiv  Detail & Related papers  (2026-08-03T01:46:19Z)
• Multi-Way Representation Alignment [19.53606443098969]
  Current strategies for mapping representations are inherently pairwise, scaling quadratically with the number of models and failing to yield a consistent global reference.<n>We first adapt Generalized Procrustes Analysis (GPA) to construct a shared universe that preserves the internal geometry essential for tasks like model stitching.<n>We then show that strict isometric alignment is suboptimal for retrieval, where agreement-maximizing methods like Canonical Correlation Analysis (CCA) typically prevail.<n>To bridge this gap, we finally propose Geometry-Corrected Procrustes Alignment (GCPA), which establishes a robust GPA-based universe followed by a post-
  arXiv  Detail & Related papers  (2026-02-05T21:33:45Z)
• Understanding and Improving UMAP with Geometric and Topological Priors: The JORC-UMAP Algorithm [1.7484982792736636]
  dimensionality reduction techniques, particularly UMAP, are widely used for visualizing high-dimensional data.<n>We introduce Ollivier-Ricci curvature as a geometric prior, reinforcing edges at geometric bottlenecks and reducing redundant links.<n>Experiments on synthetic and real-world datasets show that JORC-UMAP reduces tearing and collapse more effectively than standard UMAP and other DR methods.
  arXiv  Detail & Related papers  (2026-01-23T08:42:56Z)
• DISF: Disentangled Iterative Surface Fitting for Contact-stable Grasp Planning with Grasp Pose Alignment to the Object Center of Mass [8.61285912675644]
  We propose a novel surface fitting algorithm that integrates contact stability while preserving geometric compatibility.<n>Inspired by human grasping behavior, our method disentangles the grasp pose optimization into three sequential steps.<n>We validate our approach in simulation and real-world execution on three robot--gripper platforms.
  arXiv  Detail & Related papers  (2025-12-31T01:15:09Z)
• Geometric Operator Learning with Optimal Transport [77.16909146519227]
  We propose integrating optimal transport (OT) into operator learning for partial differential equations (PDEs) on complex geometries.<n>For 3D simulations focused on surfaces, our OT-based neural operator embeds the surface geometry into a 2D parameterized latent space.<n> Experiments with Reynolds-averaged Navier-Stokes equations (RANS) on the ShapeNet-Car and DrivAerNet-Car datasets show that our method achieves better accuracy and also reduces computational expenses.
  arXiv  Detail & Related papers  (2025-07-26T21:28:25Z)
• Dens3R: A Foundation Model for 3D Geometry Prediction [44.13431776180547]
  Dens3R is a 3D foundation model designed for joint geometric dense prediction.<n>By integrating image-pair matching features with intrinsic invariance modeling, Dens3R accurately regresses multiple geometric quantities.
  arXiv  Detail & Related papers  (2025-07-22T07:22:30Z)
• Blending 3D Geometry and Machine Learning for Multi-View Stereopsis [3.259672998844162]
  GC MVSNet plus plus is a novel approach to enforce multi-view, multi-scale supervised geometric consistency during learning.<n>This integrated GC check significantly accelerates the learning process by directly penalizing geometrically inconsistent pixels.<n>Our approach achieves a new state of the art on the DTU and BlendedMVS datasets and secures second place on the Tanks and Temples benchmark.
  arXiv  Detail & Related papers  (2025-05-06T12:22:45Z)
• Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views [45.125032766506536]
  We propose Sparse2DGS, an MVS-d Gaussian Splatting pipeline for complete and accurate reconstruction. Our key insight is to incorporate the geometric-prioritized enhancement schemes, allowing for direct and robust geometric learning under ill-posed conditions. Sparse2DGS outperforms existing methods by notable margins while being $2times$ faster than the NeRF-based fine-tuning approach.
  arXiv  Detail & Related papers  (2025-04-29T02:47:02Z)
• 3D Geometric Shape Assembly via Efficient Point Cloud Matching [59.241448711254485]
  We introduce Proxy Match Transform (PMT), an approximate high-order feature transform layer that enables reliable matching between mating surfaces of parts. Building upon PMT, we introduce a new framework, dubbed Proxy Match TransformeR (PMTR), for the geometric assembly task. We evaluate the proposed PMTR on the large-scale 3D geometric shape assembly benchmark dataset of Breaking Bad.
  arXiv  Detail & Related papers  (2024-07-15T08:50:02Z)
• Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images [56.86175251327466]
  We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context. Our approach extracts geometric context that encodes the geometric variations present in the input image and correlates depth estimation with geometric constraints. Our method unifies depth and surface normal estimations within a cohesive framework, which enables the generation of high-quality 3D geometry from images.
  arXiv  Detail & Related papers  (2024-02-08T17:57:59Z)
• Depth Completion using Geometry-Aware Embedding [22.333381291860498]
  This paper proposes an efficient method to learn geometry-aware embedding. It encodes the local and global geometric structure information from 3D points, e.g., scene layout, object's sizes and shapes, to guide dense depth estimation.
  arXiv  Detail & Related papers  (2022-03-21T12:06:27Z)
• Self-supervised Geometric Perception [96.89966337518854]
  Self-supervised geometric perception is a framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels. We show that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.
  arXiv  Detail & Related papers  (2021-03-04T15:34:43Z)
• Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking [54.58791377183574]
  Our novel hybrid recurrent multi-view stereo net consists of two core modules: 1) a light DRENet (Dense Reception Expanded) module to extract dense feature maps of original size with multi-scale context information, 2) a HU-LSTM (Hybrid U-LSTM) to regularize 3D matching volume into predicted depth map. Our method exhibits competitive performance to the state-of-the-art method while dramatically reduces memory consumption, which costs only $19.4%$ of R-MVSNet memory consumption.
  arXiv  Detail & Related papers  (2020-07-21T14:59:59Z)
• DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes [28.571946680616765]
  We propose a family of deep hierarchical convolutional networks over 3D geometric data. The first type, geodesic convolutions, defines the kernel weights over mesh surfaces or graphs. The second type, Euclidean convolutions, is independent of any underlying mesh structure.
  arXiv  Detail & Related papers  (2020-04-02T13:52:00Z)
• PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling [103.09504572409449]
  We propose a novel deep neural network based method, called PUGeo-Net, to generate uniform dense point clouds. Thanks to its geometry-centric nature, PUGeo-Net works well for both CAD models with sharp features and scanned models with rich geometric details.
  arXiv  Detail & Related papers  (2020-02-24T14:13:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.