Opening the Black Box of 3D Reconstruction Error Analysis with VECTOR

• URL: http://arxiv.org/abs/2408.03503v1
• Date: Wed, 7 Aug 2024 02:03:32 GMT
• Title: Opening the Black Box of 3D Reconstruction Error Analysis with VECTOR
• Authors: Racquel Fygenson, Kazi Jawad, Isabel Li, Francois Ayoub, Robert G. Deen, Scott Davidoff, Dominik Moritz, Mauricio Hess-Flores,
• Abstract summary: VECTOR is a visual analysis tool that improves error inspection for stereo reconstruction. VECTOR was developed in partnership with the Perseverance Mars Rover and Ingenuity Mars Helicopter terrain reconstruction team at the NASA Jet Propulsion Laboratory. We report on how this tool was used to debug and improve terrain reconstruction for the Mars 2020 mission.
• Score: 8.142689309891368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstruction of 3D scenes from 2D images is a technical challenge that impacts domains from Earth and planetary sciences and space exploration to augmented and virtual reality. Typically, reconstruction algorithms first identify common features across images and then minimize reconstruction errors after estimating the shape of the terrain. This bundle adjustment (BA) step optimizes around a single, simplifying scalar value that obfuscates many possible causes of reconstruction errors (e.g., initial estimate of the position and orientation of the camera, lighting conditions, ease of feature detection in the terrain). Reconstruction errors can lead to inaccurate scientific inferences or endanger a spacecraft exploring a remote environment. To address this challenge, we present VECTOR, a visual analysis tool that improves error inspection for stereo reconstruction BA. VECTOR provides analysts with previously unavailable visibility into feature locations, camera pose, and computed 3D points. VECTOR was developed in partnership with the Perseverance Mars Rover and Ingenuity Mars Helicopter terrain reconstruction team at the NASA Jet Propulsion Laboratory. We report on how this tool was used to debug and improve terrain reconstruction for the Mars 2020 mission.

Related papers

• Street Gaussians without 3D Object Tracker [86.62329193275916]
  Existing methods rely on labor-intensive manual labeling of object poses to reconstruct dynamic objects in canonical space and move them based on these poses during rendering. We propose a stable object tracking module by leveraging associations from 2D deep trackers within a 3D object fusion strategy. We address inevitable tracking errors by further introducing a motion learning strategy in an implicit feature space that autonomously corrects trajectory errors and recovers missed detections.
  arXiv  Detail & Related papers  (2024-12-07T05:49:42Z)
• Reconstructing Satellites in 3D from Amateur Telescope Images [42.850623200702394]
  This paper proposes a framework for the 3D reconstruction of satellites in low-Earth orbit, utilizing videos captured by small amateur telescopes. The video data obtained from these telescopes differ significantly from data for standard 3D reconstruction tasks, characterized by intense motion blur, atmospheric turbulence, pervasive background light pollution, extended focal length and constrained observational perspectives. We apply a customized Structure from Motion (SfM) approach, followed by an improved 3D Gaussian splatting algorithm, to achieve high-fidelity 3D model reconstruction.
  arXiv  Detail & Related papers  (2024-04-29T03:13:09Z)
• Indoor Obstacle Discovery on Reflective Ground via Monocular Camera [21.19387987977164]
  Visual obstacle discovery is a key step towards autonomous navigation of indoor mobile robots. In this paper, we argue that the key to this problem lies in obtaining discriminative features for reflections and obstacles. We introduce a new dataset for Obstacle on Reflective Ground (ORG), which comprises 15 scenes with various ground reflections.
  arXiv  Detail & Related papers  (2024-01-02T22:07:44Z)
• W-HMR: Monocular Human Mesh Recovery in World Space with Weak-Supervised Calibration [57.37135310143126]
  Previous methods for 3D motion recovery from monocular images often fall short due to reliance on camera coordinates. We introduce W-HMR, a weak-supervised calibration method that predicts "reasonable" focal lengths based on body distortion information. We also present the OrientCorrect module, which corrects body orientation for plausible reconstructions in world space.
  arXiv  Detail & Related papers  (2023-11-29T09:02:07Z)
• MobileBrick: Building LEGO for 3D Reconstruction on Mobile Devices [78.20154723650333]
  High-quality 3D ground-truth shapes are critical for 3D object reconstruction evaluation. We introduce a novel multi-view RGBD dataset captured using a mobile device. We obtain precise 3D ground-truth shape without relying on high-end 3D scanners.
  arXiv  Detail & Related papers  (2023-03-03T14:02:50Z)
• 3D reconstruction from spherical images: A review of techniques, applications, and prospects [2.6432771146480283]
  3D reconstruction plays an increasingly important role in modern photogrammetric systems. With the rapid evolution and extensive use of professional and consumer-grade spherical cameras, spherical images show great potential for the 3D modeling of urban and indoor scenes. This research provides a thorough survey of the state-of-the-art for 3D reconstruction of spherical images in terms of data acquisition, feature detection and matching, image orientation, and dense matching.
  arXiv  Detail & Related papers  (2023-02-09T08:45:27Z)
• Few-View Object Reconstruction with Unknown Categories and Camera Poses [80.0820650171476]
  This work explores reconstructing general real-world objects from a few images without known camera poses or object categories. The crux of our work is solving two fundamental 3D vision problems -- shape reconstruction and pose estimation. Our method FORGE predicts 3D features from each view and leverages them in conjunction with the input images to establish cross-view correspondence.
  arXiv  Detail & Related papers  (2022-12-08T18:59:02Z)
• Aerial Monocular 3D Object Detection [67.20369963664314]
  DVDET is proposed to achieve aerial monocular 3D object detection in both the 2D image space and the 3D physical space. To address the severe view deformation issue, we propose a novel trainable geo-deformable transformation module. To encourage more researchers to investigate this area, we will release the dataset and related code.
  arXiv  Detail & Related papers  (2022-08-08T08:32:56Z)
• Towards Non-Line-of-Sight Photography [48.491977359971855]
  Non-line-of-sight (NLOS) imaging is based on capturing the multi-bounce indirect reflections from the hidden objects. Active NLOS imaging systems rely on the capture of the time of flight of light through the scene. We propose a new problem formulation, called NLOS photography, to specifically address this deficiency.
  arXiv  Detail & Related papers  (2021-09-16T08:07:13Z)
• A Pose-only Solution to Visual Reconstruction and Navigation [23.86386627769292]
  Large-scale scenes and critical camera motions are great challenges facing the research community to achieve this goal. We raised a pose-only imaging geometry framework and algorithms that can help solve these challenges.
  arXiv  Detail & Related papers  (2021-03-02T07:21:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.