PointRecon: Online Point-based 3D Reconstruction via Ray-based 2D-3D Matching

• URL: http://arxiv.org/abs/2410.23245v1
• Date: Wed, 30 Oct 2024 17:29:25 GMT
• Title: PointRecon: Online Point-based 3D Reconstruction via Ray-based 2D-3D Matching
• Authors: Chen Ziwen, Zexiang Xu, Li Fuxin,
• Abstract summary: We propose a novel online, point-based 3D reconstruction method from posed monocular RGB videos. Our model maintains a global point cloud representation of the scene, continuously updating the features and 3D locations of points as new images are observed. Experiments on the ScanNet dataset show that our method achieves state-of-the-art quality among online MVS approaches.
• Score: 10.5792547614413
• License:
• Abstract: We propose a novel online, point-based 3D reconstruction method from posed monocular RGB videos. Our model maintains a global point cloud representation of the scene, continuously updating the features and 3D locations of points as new images are observed. It expands the point cloud with newly detected points while carefully removing redundancies. The point cloud updates and depth predictions for new points are achieved through a novel ray-based 2D-3D feature matching technique, which is robust against errors in previous point position predictions. In contrast to offline methods, our approach processes infinite-length sequences and provides real-time updates. Additionally, the point cloud imposes no pre-defined resolution or scene size constraints, and its unified global representation ensures view consistency across perspectives. Experiments on the ScanNet dataset show that our method achieves state-of-the-art quality among online MVS approaches. Project page: https://arthurhero.github.io/projects/pointrecon

Related papers

• Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion [53.93172686610741]
  Cross-PCC is an unsupervised point cloud completion method without requiring any 3D complete point clouds. To take advantage of the complementary information from 2D images, we use a single-view RGB image to extract 2D features. Our method even achieves comparable performance to some supervised methods.
  arXiv  Detail & Related papers  (2022-12-01T15:11:21Z)
• Learning A Locally Unified 3D Point Cloud for View Synthesis [45.757280092357355]
  We propose a new deep learning-based view synthesis paradigm that learns a locally unified 3D point cloud from source views. Experimental results on three benchmark datasets demonstrate that our method can improve the average PSNR by more than 4 dB.
  arXiv  Detail & Related papers  (2022-09-12T04:07:34Z)
• Neural 3D Scene Reconstruction with the Manhattan-world Assumption [58.90559966227361]
  This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. The proposed method outperforms previous methods by a large margin on 3D reconstruction quality.
  arXiv  Detail & Related papers  (2022-05-05T17:59:55Z)
• From Multi-View to Hollow-3D: Hallucinated Hollow-3D R-CNN for 3D Object Detection [101.20784125067559]
  We propose a new architecture, namely Hallucinated Hollow-3D R-CNN, to address the problem of 3D object detection. In our approach, we first extract the multi-view features by sequentially projecting the point clouds into the perspective view and the bird-eye view. The 3D objects are detected via a box refinement module with a novel Hierarchical Voxel RoI Pooling operation.
  arXiv  Detail & Related papers  (2021-07-30T02:00:06Z)
• How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines [49.06411148698547]
  This paper shows that a significant amount of information about the 3D scene geometry is preserved in line clouds. Our approach is based on the observation that the closest points between lines can yield a good approximation to the original 3D points.
  arXiv  Detail & Related papers  (2021-03-08T21:32:43Z)
• ParaNet: Deep Regular Representation for 3D Point Clouds [62.81379889095186]
  ParaNet is a novel end-to-end deep learning framework for representing 3D point clouds. It converts an irregular 3D point cloud into a regular 2D color image, named point geometry image (PGI) In contrast to conventional regular representation modalities based on multi-view projection and voxelization, the proposed representation is differentiable and reversible.
  arXiv  Detail & Related papers  (2020-12-05T13:19:55Z)
• RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction [19.535169371240073]
  We introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from point clouds. We decouple the instance reconstruction into global object localization and local shape prediction. Our approach consistently outperforms the state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
  arXiv  Detail & Related papers  (2020-11-30T12:58:05Z)
• GRNet: Gridding Residual Network for Dense Point Cloud Completion [54.43648460932248]
  Estimating the complete 3D point cloud from an incomplete one is a key problem in many vision and robotics applications. We propose a novel Gridding Residual Network (GRNet) for point cloud completion. Experimental results indicate that the proposed GRNet performs favorably against state-of-the-art methods on the ShapeNet, Completion3D, and KITTI benchmarks.
  arXiv  Detail & Related papers  (2020-06-06T02:46:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.