Extreme Two-View Geometry From Object Poses with Diffusion Models
- URL: http://arxiv.org/abs/2402.02800v1
- Date: Mon, 5 Feb 2024 08:18:47 GMT
- Title: Extreme Two-View Geometry From Object Poses with Diffusion Models
- Authors: Yujing Sun, Caiyi Sun, Yuan Liu, Yuexin Ma, Siu Ming Yiu
- Abstract summary: We harness the power of object priors to accurately determine two-view geometry in the face of extreme viewpoint changes.
In experiments, our method has demonstrated extraordinary robustness and resilience to large viewpoint changes.
- Score: 21.16779160086591
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Human has an incredible ability to effortlessly perceive the viewpoint
difference between two images containing the same object, even when the
viewpoint change is astonishingly vast with no co-visible regions in the
images. This remarkable skill, however, has proven to be a challenge for
existing camera pose estimation methods, which often fail when faced with large
viewpoint differences due to the lack of overlapping local features for
matching. In this paper, we aim to effectively harness the power of object
priors to accurately determine two-view geometry in the face of extreme
viewpoint changes. In our method, we first mathematically transform the
relative camera pose estimation problem to an object pose estimation problem.
Then, to estimate the object pose, we utilize the object priors learned from a
diffusion model Zero123 to synthesize novel-view images of the object. The
novel-view images are matched to determine the object pose and thus the
two-view camera pose. In experiments, our method has demonstrated extraordinary
robustness and resilience to large viewpoint changes, consistently estimating
two-view poses with exceptional generalization ability across both synthetic
and real-world datasets. Code will be available at
https://github.com/scy639/Extreme-Two-View-Geometry-From-Object-Poses-with-Diffusion-Models.
Related papers
- Generalizable Single-view Object Pose Estimation by Two-side Generating and Matching [19.730504197461144]
We present a novel generalizable object pose estimation method to determine the object pose using only one RGB image.
Our method offers generalization to unseen objects without extensive training, operates with a single reference image of the object, and eliminates the need for 3D object models or multiple views of the object.
arXiv Detail & Related papers (2024-11-24T14:31:50Z) - iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse
Views [61.707755434165335]
iFusion is a novel 3D object reconstruction framework that requires only two views with unknown camera poses.
We harness a pre-trained novel view synthesis diffusion model, which embeds implicit knowledge about the geometry and appearance of diverse objects.
Experiments demonstrate strong performance in both pose estimation and novel view synthesis.
arXiv Detail & Related papers (2023-12-28T18:59:57Z) - 3D-Aware Hypothesis & Verification for Generalizable Relative Object
Pose Estimation [69.73691477825079]
We present a new hypothesis-and-verification framework to tackle the problem of generalizable object pose estimation.
To measure reliability, we introduce a 3D-aware verification that explicitly applies 3D transformations to the 3D object representations learned from the two input images.
arXiv Detail & Related papers (2023-10-05T13:34:07Z) - Few-View Object Reconstruction with Unknown Categories and Camera Poses [80.0820650171476]
This work explores reconstructing general real-world objects from a few images without known camera poses or object categories.
The crux of our work is solving two fundamental 3D vision problems -- shape reconstruction and pose estimation.
Our method FORGE predicts 3D features from each view and leverages them in conjunction with the input images to establish cross-view correspondence.
arXiv Detail & Related papers (2022-12-08T18:59:02Z) - DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-scale
Consistency [43.09728251735362]
We present a two-step pose estimation framework to attain 6DoF object poses from 2D object bounding-boxes.
In the first step, the framework learns to segment objects from real and synthetic data.
In the second step, we design a dual-scale pose estimation network, namely DSC-PoseNet.
Our method outperforms state-of-the-art models trained on synthetic data by a large margin.
arXiv Detail & Related papers (2021-04-08T10:19:35Z) - CosyPose: Consistent multi-view multi-object 6D pose estimation [48.097599674329004]
We present a single-view single-object 6D pose estimation method, which we use to generate 6D object pose hypotheses.
Second, we develop a robust method for matching individual 6D object pose hypotheses across different input images.
Third, we develop a method for global scene refinement given multiple object hypotheses and their correspondences across views.
arXiv Detail & Related papers (2020-08-19T14:11:56Z) - Single View Metrology in the Wild [94.7005246862618]
We present a novel approach to single view metrology that can recover the absolute scale of a scene represented by 3D heights of objects or camera height above the ground.
Our method relies on data-driven priors learned by a deep network specifically designed to imbibe weakly supervised constraints from the interplay of the unknown camera with 3D entities such as object heights.
We demonstrate state-of-the-art qualitative and quantitative results on several datasets as well as applications including virtual object insertion.
arXiv Detail & Related papers (2020-07-18T22:31:33Z) - Object-Centric Image Generation from Layouts [93.10217725729468]
We develop a layout-to-image-generation method to generate complex scenes with multiple objects.
Our method learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity.
We introduce SceneFID, an object-centric adaptation of the popular Fr'echet Inception Distance metric, that is better suited for multi-object images.
arXiv Detail & Related papers (2020-03-16T21:40:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.