Multi-View Stereo Representation Revisit: Region-Aware MVSNet

• URL: http://arxiv.org/abs/2304.13614v2
• Date: Thu, 27 Apr 2023 07:08:37 GMT
• Title: Multi-View Stereo Representation Revisit: Region-Aware MVSNet
• Authors: Yisu Zhang and Jianke Zhu and Lixiang Lin
• Abstract summary: Deep learning-based multi-view stereo has emerged as a powerful paradigm for reconstructing the complete geometrically-detailed objects from multi-views. We propose RA-MVSNet to take advantage of point-to-surface distance so that the model is able to perceive a wider range of surfaces. Our proposed RA-MVSNet is patch-awared, since the perception range is enhanced by associating hypothetical planes with a patch of surface.
• Score: 8.264851594332677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning-based multi-view stereo has emerged as a powerful paradigm for reconstructing the complete geometrically-detailed objects from multi-views. Most of the existing approaches only estimate the pixel-wise depth value by minimizing the gap between the predicted point and the intersection of ray and surface, which usually ignore the surface topology. It is essential to the textureless regions and surface boundary that cannot be properly reconstructed. To address this issue, we suggest to take advantage of point-to-surface distance so that the model is able to perceive a wider range of surfaces. To this end, we predict the distance volume from cost volume to estimate the signed distance of points around the surface. Our proposed RA-MVSNet is patch-awared, since the perception range is enhanced by associating hypothetical planes with a patch of surface. Therefore, it could increase the completion of textureless regions and reduce the outliers at the boundary. Moreover, the mesh topologies with fine details can be generated by the introduced distance volume. Comparing to the conventional deep learning-based multi-view stereo methods, our proposed RA-MVSNet approach obtains more complete reconstruction results by taking advantage of signed distance supervision. The experiments on both the DTU and Tanks \& Temples datasets demonstrate that our proposed approach achieves the state-of-the-art results.

Related papers

• Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction [29.26047429432185]
  SuRF is a new framework that incorporates a new Region sparsification based on a matching Field. To our knowledge, this is the first unsupervised method achieving end-to-end sparsification. Experiments show that our reconstructions exhibit high-quality details and achieve new state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-09-05T15:48:02Z)
• Arbitrary-Scale Point Cloud Upsampling by Voxel-Based Network with Latent Geometric-Consistent Learning [52.825441454264585]
  We propose an arbitrary-scale Point cloud Upsampling framework using Voxel-based Network (textbfPU-VoxelNet) Thanks to the completeness and regularity inherited from the voxel representation, voxel-based networks are capable of providing predefined grid space to approximate 3D surface. A density-guided grid resampling method is developed to generate high-fidelity points while effectively avoiding sampling outliers.
  arXiv  Detail & Related papers  (2024-03-08T07:31:14Z)
• V-FUSE: Volumetric Depth Map Fusion with Long-Range Constraints [6.7197802356130465]
  We introduce a learning-based depth map fusion framework that accepts a set of depth and confidence maps generated by a Multi-View Stereo (MVS) algorithm as input and improves them. We also introduce a depth search window estimation sub-network trained jointly with the larger fusion sub-network to reduce the depth hypothesis search space along each ray. Our method learns to model depth consensus and violations of visibility constraints directly from the data.
  arXiv  Detail & Related papers  (2023-08-17T00:39:56Z)
• C2F2NeUS: Cascade Cost Frustum Fusion for High Fidelity and Generalizable Neural Surface Reconstruction [12.621233209149953]
  We introduce a novel integration scheme that combines the multi-view stereo with neural signed distance function representations. Our method reconstructs robust surfaces and outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-16T17:56:16Z)
• Visual Attention-based Self-supervised Absolute Depth Estimation using Geometric Priors in Autonomous Driving [8.045833295463094]
  We introduce a fully Visual Attention-based Depth (VADepth) network, where spatial attention and channel attention are applied to all stages. By continuously extracting the dependencies of features along the spatial and channel dimensions over a long distance, VADepth network can effectively preserve important details. Experimental results on the KITTI dataset show that this architecture achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-18T08:01:38Z)
• Improving Monocular Visual Odometry Using Learned Depth [84.05081552443693]
  We propose a framework to exploit monocular depth estimation for improving visual odometry (VO) The core of our framework is a monocular depth estimation module with a strong generalization capability for diverse scenes. Compared with current learning-based VO methods, our method demonstrates a stronger generalization ability to diverse scenes.
  arXiv  Detail & Related papers  (2022-04-04T06:26:46Z)
• Uncertainty-Aware Deep Multi-View Photometric Stereo [100.97116470055273]
  Photometric stereo (PS) is excellent at recovering high-frequency surface details, whereas multi-view stereo (MVS) can help remove the low-frequency distortion due to PS and retain the global shape. This paper proposes an approach that can effectively utilize such complementary strengths of PS and MVS. We estimate per-pixel surface normals and depth using an uncertainty-aware deep-PS network and deep-MVS network, respectively.
  arXiv  Detail & Related papers  (2022-02-26T05:45:52Z)
• Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
  We present a modern solution to the multi-view photometric stereo problem (MVPS) We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry. Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
  arXiv  Detail & Related papers  (2021-10-11T20:20:03Z)
• Learning Signed Distance Field for Multi-view Surface Reconstruction [24.090786783370195]
  We introduce a novel neural surface reconstruction framework that leverages the knowledge of stereo matching and feature consistency. We apply a signed distance field (SDF) and a surface light field to represent the scene geometry and appearance respectively. Our method is able to improve the robustness of geometry estimation and support reconstruction of complex scene topologies.
  arXiv  Detail & Related papers  (2021-08-23T06:23:50Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)
• OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems [88.41004332322788]
  We present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estimation. We integrate our omnidirectional depth estimates into the visual odometry (VO) and add a loop closing module for global consistency.
  arXiv  Detail & Related papers  (2020-03-18T05:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.