SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsis

• URL: http://arxiv.org/abs/2005.12690v1
• Date: Tue, 26 May 2020 13:13:02 GMT
• Title: SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsis
• Authors: Mengqi Ji, Jinzhi Zhang, Qionghai Dai, Lu Fang
• Abstract summary: Multi-view stereopsis (MVS) tries to recover the 3D model from 2D images. We investigate sparse-MVS with large baseline angles since the sparser sensation is more practical and more cost-efficient. We present SurfaceNet+, a volumetric method to handle the 'incompleteness' and the 'inaccuracy' problems induced by a very sparse MVS setup.
• Score: 52.35697180864202
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-view stereopsis (MVS) tries to recover the 3D model from 2D images. As the observations become sparser, the significant 3D information loss makes the MVS problem more challenging. Instead of only focusing on densely sampled conditions, we investigate sparse-MVS with large baseline angles since the sparser sensation is more practical and more cost-efficient. By investigating various observation sparsities, we show that the classical depth-fusion pipeline becomes powerless for the case with a larger baseline angle that worsens the photo-consistency check. As another line of the solution, we present SurfaceNet+, a volumetric method to handle the 'incompleteness' and the 'inaccuracy' problems induced by a very sparse MVS setup. Specifically, the former problem is handled by a novel volume-wise view selection approach. It owns superiority in selecting valid views while discarding invalid occluded views by considering the geometric prior. Furthermore, the latter problem is handled via a multi-scale strategy that consequently refines the recovered geometry around the region with the repeating pattern. The experiments demonstrate the tremendous performance gap between SurfaceNet+ and state-of-the-art methods in terms of precision and recall. Under the extreme sparse-MVS settings in two datasets, where existing methods can only return very few points, SurfaceNet+ still works as well as in the dense MVS setting. The benchmark and the implementation are publicly available at https://github.com/mjiUST/SurfaceNet-plus.

Related papers

• GLACE: Global Local Accelerated Coordinate Encoding [66.87005863868181]
  Scene coordinate regression methods are effective in small-scale scenes but face significant challenges in large-scale scenes. We propose GLACE, which integrates pre-trained global and local encodings and enables SCR to scale to large scenes with only a single small-sized network. Our method achieves state-of-the-art results on large-scale scenes with a low-map-size model.
  arXiv  Detail & Related papers  (2024-06-06T17:59:50Z)
• S-VolSDF: Sparse Multi-View Stereo Regularization of Neural Implicit Surfaces [75.30792581941789]
  Neural rendering of implicit surfaces performs well in 3D vision applications. When only sparse input images are available, output quality drops significantly due to the shape-radiance ambiguity problem. We propose to regularize neural rendering optimization with an MVS solution.
  arXiv  Detail & Related papers  (2023-03-30T21:10:58Z)
• Scatter Points in Space: 3D Detection from Multi-view Monocular Images [8.71944437852952]
  3D object detection from monocular image(s) is a challenging and long-standing problem of computer vision. Recent methods tend to aggregate multiview feature by sampling regular 3D grid densely in space. We propose a learnable keypoints sampling method, which scatters pseudo surface points in 3D space, in order to keep data sparsity.
  arXiv  Detail & Related papers  (2022-08-31T09:38:05Z)
• Neural 3D Scene Reconstruction with the Manhattan-world Assumption [58.90559966227361]
  This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. The proposed method outperforms previous methods by a large margin on 3D reconstruction quality.
  arXiv  Detail & Related papers  (2022-05-05T17:59:55Z)
• RC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering [16.679446000660654]
  We propose a novel approach with neural rendering (RC-MVSNet) to solve ambiguity issues of correspondences among views. Specifically, we impose a depth rendering consistency loss to constrain the geometry features close to the object surface. We also introduce a reference view loss to generate consistent supervision, even for non-Lambertian surfaces.
  arXiv  Detail & Related papers  (2022-03-08T09:24:05Z)
• 3DVNet: Multi-View Depth Prediction and Volumetric Refinement [68.68537312256144]
  3DVNet is a novel multi-view stereo (MVS) depth-prediction method. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions. We show that our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics.
  arXiv  Detail & Related papers  (2021-12-01T00:52:42Z)
• Recurrent Multi-view Alignment Network for Unsupervised Surface Registration [79.72086524370819]
  Learning non-rigid registration in an end-to-end manner is challenging due to the inherent high degrees of freedom and the lack of labeled training data. We propose to represent the non-rigid transformation with a point-wise combination of several rigid transformations. We also introduce a differentiable loss function that measures the 3D shape similarity on the projected multi-view 2D depth images.
  arXiv  Detail & Related papers  (2020-11-24T14:22:42Z)
• M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Network [13.447649324253572]
  We propose a novel unsupervised multi-metric MVS network, named M3VSNet, for dense point cloud reconstruction without supervision. To improve the robustness and completeness of point cloud reconstruction, we propose a novel multi-metric loss function that combines pixel-wise and feature-wise loss function. Experimental results show that M3VSNet establishes the state-of-the-arts unsupervised method and achieves comparable performance with previous supervised MVSNet.
  arXiv  Detail & Related papers  (2020-04-30T09:26:51Z)
• M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Network [13.447649324253572]
  We propose a novel unsupervised multi-metric MVS network, named M3VSNet, for dense point cloud reconstruction without supervision. To improve the robustness and completeness of point cloud reconstruction, we propose a novel multi-metric loss function that combines pixel-wise and feature-wise loss function. Experimental results show that M3VSNet establishes the state-of-the-arts unsupervised method and achieves comparable performance with previous supervised MVSNet.
  arXiv  Detail & Related papers  (2020-04-21T02:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.