Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure
Reconstruction from an RGB Video
- URL: http://arxiv.org/abs/2005.03372v3
- Date: Wed, 20 May 2020 04:57:24 GMT
- Title: Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure
Reconstruction from an RGB Video
- Authors: Peng Wang, Lingjie Liu, Nenglun Chen, Hung-Kuo Chu, Christian
Theobalt, Wenping Wang
- Abstract summary: Thin structures, such as wire-frame sculptures, fences, cables, power lines, and tree branches, are common in the real world.
It is extremely challenging to acquire their 3D digital models using traditional image-based or depth-based reconstruction methods because thin structures often lack distinct point features and have severe self-occlusion.
We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera.
- Score: 90.93141123721713
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Thin structures, such as wire-frame sculptures, fences, cables, power lines,
and tree branches, are common in the real world. It is extremely challenging to
acquire their 3D digital models using traditional image-based or depth-based
reconstruction methods because thin structures often lack distinct point
features and have severe self-occlusion. We propose the first approach that
simultaneously estimates camera motion and reconstructs the geometry of complex
3D thin structures in high quality from a color video captured by a handheld
camera. Specifically, we present a new curve-based approach to estimate
accurate camera poses by establishing correspondences between featureless thin
objects in the foreground in consecutive video frames, without requiring visual
texture in the background scene to lock on. Enabled by this effective
curve-based camera pose estimation strategy, we develop an iterative
optimization method with tailored measures on geometry, topology as well as
self-occlusion handling for reconstructing 3D thin structures. Extensive
validations on a variety of thin structures show that our method achieves
accurate camera pose estimation and faithful reconstruction of 3D thin
structures with complex shape and topology at a level that has not been
attained by other existing reconstruction methods.
Related papers
- FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views [93.6881532277553]
We present FLARE, a feed-forward model designed to infer high-quality camera poses and 3D geometry from uncalibrated sparse-view images.
Our solution features a cascaded learning paradigm with camera pose serving as the critical bridge, recognizing its essential role in mapping 3D structures onto 2D image planes.
arXiv Detail & Related papers (2025-02-17T18:54:05Z) - Unified Few-shot Crack Segmentation and its Precise 3D Automatic Measurement in Concrete Structures [2.178830801484721]
This study introduces a framework for two-dimensional (2D) crack detection, three-dimensional (3D) reconstruction, and 3D automatic crack measurement.
We developed a crack segmentation method with strong generalization across unfamiliar scenarios, enabling the generation of precise 2D crack masks.
By leveraging both image- and LiDAR-SLAM, we developed a multi-frame and multi-modal fusion framework that produces dense, colorized point clouds.
arXiv Detail & Related papers (2025-01-15T23:36:05Z) - R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras [106.52409577316389]
R3D3 is a multi-camera system for dense 3D reconstruction and ego-motion estimation.
Our approach exploits spatial-temporal information from multiple cameras, and monocular depth refinement.
We show that this design enables a dense, consistent 3D reconstruction of challenging, dynamic outdoor environments.
arXiv Detail & Related papers (2023-08-28T17:13:49Z) - FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
We propose a novel test-time optimization approach for 3D scene reconstruction.
Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
arXiv Detail & Related papers (2023-08-10T17:55:02Z) - LIST: Learning Implicitly from Spatial Transformers for Single-View 3D
Reconstruction [5.107705550575662]
List is a novel neural architecture that leverages local and global image features to reconstruct geometric and topological structure of a 3D object from a single image.
We show the superiority of our model in reconstructing 3D objects from both synthetic and real-world images against the state of the art.
arXiv Detail & Related papers (2023-07-23T01:01:27Z) - Multi-View Neural Surface Reconstruction with Structured Light [7.709526244898887]
Three-dimensional (3D) object reconstruction based on differentiable rendering (DR) is an active research topic in computer vision.
We introduce active sensing with structured light (SL) into multi-view 3D object reconstruction based on DR to learn the unknown geometry and appearance of arbitrary scenes and camera poses.
Our method realizes high reconstruction accuracy in the textureless region and reduces efforts for camera pose calibration.
arXiv Detail & Related papers (2022-11-22T03:10:46Z) - Towards Non-Line-of-Sight Photography [48.491977359971855]
Non-line-of-sight (NLOS) imaging is based on capturing the multi-bounce indirect reflections from the hidden objects.
Active NLOS imaging systems rely on the capture of the time of flight of light through the scene.
We propose a new problem formulation, called NLOS photography, to specifically address this deficiency.
arXiv Detail & Related papers (2021-09-16T08:07:13Z) - Online Adaptation for Consistent Mesh Reconstruction in the Wild [147.22708151409765]
We pose video-based reconstruction as a self-supervised online adaptation problem applied to any incoming test video.
We demonstrate that our algorithm recovers temporally consistent and reliable 3D structures from videos of non-rigid objects including those of animals captured in the wild.
arXiv Detail & Related papers (2020-12-06T07:22:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.