Stereo Unstructured Magnification: Multiple Homography Image for View
Synthesis
- URL: http://arxiv.org/abs/2204.00156v1
- Date: Fri, 1 Apr 2022 01:39:28 GMT
- Title: Stereo Unstructured Magnification: Multiple Homography Image for View
Synthesis
- Authors: Qi Zhang and Xin Huang and Ying Feng and Xue Wang and Hongdong Li and
Qing Wang
- Abstract summary: We study the problem of view synthesis with certain amount of rotations from a pair of images, what we called stereo unstructured magnification.
We propose a novel multiple homography image representation, comprising of a set of scene planes with fixed normals and distances.
We derive an angle-based cost to guide the blending of multi-normal images by exploiting per-normal geometry.
- Score: 72.09193030350396
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper studies the problem of view synthesis with certain amount of
rotations from a pair of images, what we called stereo unstructured
magnification. While the multi-plane image representation is well suited for
view synthesis with depth invariant, how to generalize it to unstructured views
remains a significant challenge. This is primarily due to the depth-dependency
caused by camera frontal parallel representation. Here we propose a novel
multiple homography image (MHI) representation, comprising of a set of scene
planes with fixed normals and distances. A two-stage network is developed for
novel view synthesis. Stage-1 is an MHI reconstruction module that predicts the
MHIs and composites layered multi-normal images along the normal direction.
Stage-2 is a normal-blending module to find blending weights. We also derive an
angle-based cost to guide the blending of multi-normal images by exploiting
per-normal geometry. Compared with the state-of-the-art methods, our method
achieves superior performance for view synthesis qualitatively and
quantitatively, especially for cases when the cameras undergo rotations.
Related papers
- MultiDiff: Consistent Novel View Synthesis from a Single Image [60.04215655745264]
MultiDiff is a novel approach for consistent novel view synthesis of scenes from a single RGB image.
Our results demonstrate that MultiDiff outperforms state-of-the-art methods on the challenging, real-world datasets RealEstate10K and ScanNet.
arXiv Detail & Related papers (2024-06-26T17:53:51Z) - DUSt3R: Geometric 3D Vision Made Easy [8.471330244002564]
We introduce DUSt3R, a novel paradigm for Dense and Unconstrained Stereo 3D Reconstruction of arbitrary image collections.
We show that this formulation smoothly unifies the monocular and binocular reconstruction cases.
Our formulation directly provides a 3D model of the scene as well as depth information, but interestingly, we can seamlessly recover from it, pixel matches, relative and absolute camera.
arXiv Detail & Related papers (2023-12-21T18:52:14Z) - SAMPLING: Scene-adaptive Hierarchical Multiplane Images Representation
for Novel View Synthesis from a Single Image [60.52991173059486]
We introduce SAMPLING, a Scene-adaptive Hierarchical Multiplane Images Representation for Novel View Synthesis from a Single Image.
Our method demonstrates considerable performance gains in large-scale unbounded outdoor scenes using a single image on the KITTI dataset.
arXiv Detail & Related papers (2023-09-12T15:33:09Z) - Single-View View Synthesis with Self-Rectified Pseudo-Stereo [49.946151180828465]
We leverage the reliable and explicit stereo prior to generate a pseudo-stereo viewpoint.
We propose a self-rectified stereo synthesis to amend erroneous regions in an identify-rectify manner.
Our method outperforms state-of-the-art single-view view synthesis methods and stereo synthesis methods.
arXiv Detail & Related papers (2023-04-19T09:36:13Z) - Single-View View Synthesis in the Wild with Learned Adaptive Multiplane
Images [15.614631883233898]
Existing methods have shown promising results leveraging monocular depth estimation and color inpainting with layered depth representations.
We propose a new method based on the multiplane image (MPI) representation.
The experiments on both synthetic and real datasets demonstrate that our trained model works remarkably well and achieves state-of-the-art results.
arXiv Detail & Related papers (2022-05-24T02:57:16Z) - Practical Wide-Angle Portraits Correction with Deep Structured Models [17.62752136436382]
This paper introduces the first deep learning based approach to remove perspective distortions from photos.
Given a wide-angle portrait as input, we build a cascaded network consisting of a LineNet, a ShapeNet, and a transition module.
For the quantitative evaluation, we introduce two novel metrics, line consistency and face congruence.
arXiv Detail & Related papers (2021-04-26T10:47:35Z) - Deep Two-View Structure-from-Motion Revisited [83.93809929963969]
Two-view structure-from-motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM.
We propose to revisit the problem of deep two-view SfM by leveraging the well-posedness of the classic pipeline.
Our method consists of 1) an optical flow estimation network that predicts dense correspondences between two frames; 2) a normalized pose estimation module that computes relative camera poses from the 2D optical flow correspondences, and 3) a scale-invariant depth estimation network that leverages epipolar geometry to reduce the search space, refine the dense correspondences, and estimate relative depth maps.
arXiv Detail & Related papers (2021-04-01T15:31:20Z) - Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics [72.9038524082252]
We propose a compact single-shot monocular hyperspectral-depth (HS-D) imaging method.
Our method uses a diffractive optical element (DOE), the point spread function of which changes with respect to both depth and spectrum.
To facilitate learning the DOE, we present a first HS-D dataset by building a benchtop HS-D imager.
arXiv Detail & Related papers (2020-09-01T14:19:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.