Sensing Surface Patches in Volume Rendering for Inferring Signed Distance Functions

• URL: http://arxiv.org/abs/2412.16467v1
• Date: Sat, 21 Dec 2024 03:49:29 GMT
• Title: Sensing Surface Patches in Volume Rendering for Inferring Signed Distance Functions
• Authors: Sijia Jiang, Tong Wu, Jing Hua, Zhizhong Han,
• Abstract summary: It is vital to recover 3D geometry from multi-view RGB images in many 3D computer vision tasks. We introduce a method to infer signed distance functions with a better sense of surfaces through volume rendering.
• Score: 26.38988582983954
• License:
• Abstract: It is vital to recover 3D geometry from multi-view RGB images in many 3D computer vision tasks. The latest methods infer the geometry represented as a signed distance field by minimizing the rendering error on the field through volume rendering. However, it is still challenging to explicitly impose constraints on surfaces for inferring more geometry details due to the limited ability of sensing surfaces in volume rendering. To resolve this problem, we introduce a method to infer signed distance functions (SDFs) with a better sense of surfaces through volume rendering. Using the gradients and signed distances, we establish a small surface patch centered at the estimated intersection along a ray by pulling points randomly sampled nearby. Hence, we are able to explicitly impose surface constraints on the sensed surface patch, such as multi-view photo consistency and supervision from depth or normal priors, through volume rendering. We evaluate our method by numerical and visual comparisons on scene benchmarks. Our superiority over the latest methods justifies our effectiveness.

Related papers

• Multi-View Stereo Representation Revisit: Region-Aware MVSNet [8.264851594332677]
  Deep learning-based multi-view stereo has emerged as a powerful paradigm for reconstructing the complete geometrically-detailed objects from multi-views. We propose RA-MVSNet to take advantage of point-to-surface distance so that the model is able to perceive a wider range of surfaces. Our proposed RA-MVSNet is patch-awared, since the perception range is enhanced by associating hypothetical planes with a patch of surface.
  arXiv  Detail & Related papers  (2023-04-26T15:17:51Z)
• NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images [17.637064969966847]
  NeAT is a new neural rendering framework that learns implicit surfaces with arbitrary topologies from multi-view images. NeAT supports easy field-to-mesh conversion using the classic Marching Cubes algorithm. Our approach is able to faithfully reconstruct both watertight and non-watertight surfaces.
  arXiv  Detail & Related papers  (2023-03-21T16:49:41Z)
• Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement [78.48648360358193]
  We present a novel framework that generates textured surface meshes from images. Our approach begins by efficiently initializing the geometry and view-dependency appearance with a NeRF. We jointly refine the appearance with geometry and bake it into texture images for real-time rendering.
  arXiv  Detail & Related papers  (2023-03-03T17:14:44Z)
• CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization [54.69408516025872]
  CAP-UDF is a novel method to learn consistency-aware UDF from raw point clouds. We train a neural network to gradually infer the relationship between queries and the approximated surface. We also introduce a polygonization algorithm to extract surfaces using the gradients of the learned UDF.
  arXiv  Detail & Related papers  (2022-10-06T08:51:08Z)
• Scatter Points in Space: 3D Detection from Multi-view Monocular Images [8.71944437852952]
  3D object detection from monocular image(s) is a challenging and long-standing problem of computer vision. Recent methods tend to aggregate multiview feature by sampling regular 3D grid densely in space. We propose a learnable keypoints sampling method, which scatters pseudo surface points in 3D space, in order to keep data sparsity.
  arXiv  Detail & Related papers  (2022-08-31T09:38:05Z)
• Neural 3D Scene Reconstruction with the Manhattan-world Assumption [58.90559966227361]
  This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. The proposed method outperforms previous methods by a large margin on 3D reconstruction quality.
  arXiv  Detail & Related papers  (2022-05-05T17:59:55Z)
• POCO: Point Convolution for Surface Reconstruction [92.22371813519003]
  Implicit neural networks have been successfully used for surface reconstruction from point clouds. Many of them face scalability issues as they encode the isosurface function of a whole object or scene into a single latent vector. We propose to use point cloud convolutions and compute latent vectors at each input point.
  arXiv  Detail & Related papers  (2022-01-05T21:26:18Z)
• Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis [4.597864989500202]
  We propose a new method for reconstructing implicit 3D human models from sparse multi-view RGB videos. Our method defines the neural scene representation on the mesh surface points and signed distances from the surface of a human body mesh.
  arXiv  Detail & Related papers  (2022-01-05T16:25:32Z)
• MeshUDF: Fast and Differentiable Meshing of Unsigned Distance Field Networks [68.82901764109685]
  Recent work modelling 3D open surfaces train deep neural networks to approximate Unsigned Distance Fields (UDFs) We propose to directly mesh deep UDFs as open surfaces with an extension of marching cubes, by locally detecting surface crossings. Our method is order of magnitude faster than meshing a dense point cloud, and more accurate than inflating open surfaces.
  arXiv  Detail & Related papers  (2021-11-29T14:24:02Z)
• Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images [64.53227129573293]
  We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. We design neural networks capable of generating high-quality parametric 3D surfaces which are consistent between views. Our method is supervised and trained on a public dataset of shapes from common object categories.
  arXiv  Detail & Related papers  (2020-08-18T06:33:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.