IM360: Textured Mesh Reconstruction for Large-scale Indoor Mapping with 360$^\circ$ Cameras

• URL: http://arxiv.org/abs/2502.12545v2
• Date: Wed, 19 Feb 2025 18:52:15 GMT
• Title: IM360: Textured Mesh Reconstruction for Large-scale Indoor Mapping with 360$^\circ$ Cameras
• Authors: Dongki Jung, Jaehoon Choi, Yonghan Lee, Dinesh Manocha,
• Abstract summary: We present a novel 3D reconstruction pipeline for 360$circ$ cameras for 3D mapping and rendering of indoor environments. Our approach (IM360) leverages the wide field of view of omnidirectional images and integrates the spherical camera model into every core component of the SfM pipeline. We evaluate our pipeline on large-scale indoor scenes from the Matterport3D and Stanford2D3D datasets.
• Score: 53.53895891356167
• License:
• Abstract: We present a novel 3D reconstruction pipeline for 360$^\circ$ cameras for 3D mapping and rendering of indoor environments. Traditional Structure-from-Motion (SfM) methods may not work well in large-scale indoor scenes due to the prevalence of textureless and repetitive regions. To overcome these challenges, our approach (IM360) leverages the wide field of view of omnidirectional images and integrates the spherical camera model into every core component of the SfM pipeline. In order to develop a comprehensive 3D reconstruction solution, we integrate a neural implicit surface reconstruction technique to generate high-quality surfaces from sparse input data. Additionally, we utilize a mesh-based neural rendering approach to refine texture maps and accurately capture view-dependent properties by combining diffuse and specular components. We evaluate our pipeline on large-scale indoor scenes from the Matterport3D and Stanford2D3D datasets. In practice, IM360 demonstrate superior performance in terms of textured mesh reconstruction over SOTA. We observe accuracy improvements in terms of camera localization and registration as well as rendering high frequency details.

Related papers

• 3D Gaussian Splatting with Normal Information for Mesh Extraction and Improved Rendering [8.59572577251833]
  We propose a novel regularization method using the gradients of a signed distance function estimated from the Gaussians. We demonstrate the effectiveness of our approach on datasets such as Mip-NeRF360, Tanks and Temples, and Deep-Blending.
  arXiv  Detail & Related papers  (2025-01-14T18:40:33Z)
• 360Recon: An Accurate Reconstruction Method Based on Depth Fusion from 360 Images [10.564434148892362]
  360-degree images offer a significantly wider field of view compared to traditional pinhole cameras. This makes them crucial for applications in VR, AR, and related fields. We propose 360Recon, an innovative MVS algorithm for ERP images.
  arXiv  Detail & Related papers  (2024-11-28T12:30:45Z)
• Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priors [51.36238367193988]
  We tackle sparse-view reconstruction of a 360 3D scene using priors from latent diffusion models (LDM) We present SparseSplat360, a method that employs a cascade of in-painting and artifact removal models to fill in missing details and clean novel views. Our method generates entire 360 scenes from as few as 9 input views, with a high degree of foreground and background detail.
  arXiv  Detail & Related papers  (2024-05-26T11:01:39Z)
• RoGUENeRF: A Robust Geometry-Consistent Universal Enhancer for NeRF [1.828790674925926]
  2D enhancers can be pre-trained to recover some detail but are agnostic to scene geometry. Existing 3D enhancers are able to transfer detail from nearby training images in a generalizable manner. We propose a neural rendering enhancer, RoGUENeRF, which exploits the best of both paradigms.
  arXiv  Detail & Related papers  (2024-03-18T16:11:42Z)
• ReconFusion: 3D Reconstruction with Diffusion Priors [104.73604630145847]
  We present ReconFusion to reconstruct real-world scenes using only a few photos. Our approach leverages a diffusion prior for novel view synthesis, trained on synthetic and multiview datasets. Our method synthesizes realistic geometry and texture in underconstrained regions while preserving the appearance of observed regions.
  arXiv  Detail & Related papers  (2023-12-05T18:59:58Z)
• EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction [53.28220984270622]
  3D reconstruction methods should generate high-fidelity results with 3D consistency in real-time. Our method can reconstruct high-quality appearance and accurate mesh on both synthetic and real-world datasets. Our method can be trained in just 1-2 hours using a single GPU and run on mobile devices at over 40 FPS (Frames Per Second)
  arXiv  Detail & Related papers  (2023-11-16T11:30:56Z)
• TMO: Textured Mesh Acquisition of Objects with a Mobile Device by using Differentiable Rendering [54.35405028643051]
  We present a new pipeline for acquiring a textured mesh in the wild with a single smartphone. Our method first introduces an RGBD-aided structure from motion, which can yield filtered depth maps. We adopt the neural implicit surface reconstruction method, which allows for high-quality mesh.
  arXiv  Detail & Related papers  (2023-03-27T10:07:52Z)
• High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
  We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
  arXiv  Detail & Related papers  (2022-11-28T18:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.