Surface Normal Estimation of Tilted Images via Spatial Rectifier
- URL: http://arxiv.org/abs/2007.09264v2
- Date: Thu, 14 Jul 2022 17:43:20 GMT
- Title: Surface Normal Estimation of Tilted Images via Spatial Rectifier
- Abstract summary: Existing approaches exhibit bounded performance on predicting surface normals because they were trained using gravity-aligned images.
Our two main hypotheses are: (1) visual scene layout is indicative of the gravity direction; and (2) not all surfaces are equally represented by a learned estimator due to the structured distribution of the training data.
We design a spatial estimator that is learned to transform the surface normal distribution of a tilted image to the one that matches the gravity-aligned data distribution.
- Score: 23.112310775824714
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we present a spatial rectifier to estimate surface normals of
tilted images. Tilted images are of particular interest as more visual data are
captured by arbitrarily oriented sensors such as body-/robot-mounted cameras.
Existing approaches exhibit bounded performance on predicting surface normals
because they were trained using gravity-aligned images. Our two main hypotheses
are: (1) visual scene layout is indicative of the gravity direction; and (2)
not all surfaces are equally represented by a learned estimator due to the
structured distribution of the training data, thus, there exists a
transformation for each tilted image that is more responsive to the learned
estimator than others. We design a spatial rectifier that is learned to
transform the surface normal distribution of a tilted image to the rectified
one that matches the gravity-aligned training data distribution. Along with the
spatial rectifier, we propose a novel truncated angular loss that offers a
stronger gradient at smaller angular errors and robustness to outliers. The
resulting estimator outperforms the state-of-the-art methods including data
augmentation baselines not only on ScanNet and NYUv2 but also on a new dataset
called Tilt-RGBD that includes considerable roll and pitch camera motion.
Related papers
- Visual Relocalization from Sparse Views in Aliased and Low-Texture Environments via Novel View Synthesis [17.82023714307905]
We propose a visual relocalization method that departs from classical correspondence-based pipelines.<n>Our key contribution is a geometry-aware training strategy that combines photometric and geometric losses.<n>We show that this joint optimization produces a 3DGS model that better fits the underlying scene geometry.
arXiv Detail & Related papers (2026-07-24T09:46:07Z) - VA-GS: Enhancing the Geometric Representation of Gaussian Splatting via View Alignment [48.147381011235446]
3D Gaussian Splatting has recently emerged as an efficient solution for real-time novel view synthesis.<n>We propose a novel method that enhances the geometric representation of 3D Gaussians through view alignment.<n>Our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis.
arXiv Detail & Related papers (2025-10-13T14:44:50Z) - Loc$^2$: Interpretable Cross-View Localization via Depth-Lifted Local Feature Matching [80.57282092735991]
We propose an accurate and interpretable fine-grained cross-view localization method.<n>It estimates the 3 Degrees of Freedom (DoF) pose of a ground-level image by matching its local features with a reference aerial image.<n> Experiments show state-of-the-art accuracy in challenging scenarios such as cross-area testing and unknown orientation.
arXiv Detail & Related papers (2025-09-11T18:52:16Z) - Pose-free 3D Gaussian splatting via shape-ray estimation [19.489216159648404]
We introduce SHARE, a pose-free, feed-forward Gaussian splatting framework.<n>It overcomes ambiguities by joint shape and camera rays estimation.<n>Our method achieves robust performance in pose-free generalizable Gaussian splatting.
arXiv Detail & Related papers (2025-05-29T01:34:40Z) - Vanishing Point Estimation in Uncalibrated Images with Prior Gravity
Direction [82.72686460985297]
We tackle the problem of estimating a Manhattan frame.
We derive two new 2-line solvers, one of which does not suffer from singularities affecting existing solvers.
We also design a new non-minimal method, running on an arbitrary number of lines, to boost the performance in local optimization.
arXiv Detail & Related papers (2023-08-21T13:03:25Z) - View Consistent Purification for Accurate Cross-View Localization [59.48131378244399]
This paper proposes a fine-grained self-localization method for outdoor robotics.
The proposed method addresses limitations in existing cross-view localization methods.
It is the first sparse visual-only method that enhances perception in dynamic environments.
arXiv Detail & Related papers (2023-08-16T02:51:52Z) - Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via
Geometry-Guided Cross-View Transformer [66.82008165644892]
We propose a method to increase the accuracy of a ground camera's location and orientation by estimating the relative rotation and translation between the ground-level image and its matched/retrieved satellite image.
Experimental results demonstrate that our method significantly outperforms the state-of-the-art.
arXiv Detail & Related papers (2023-07-16T11:52:27Z) - 2D LiDAR and Camera Fusion Using Motion Cues for Indoor Layout
Estimation [2.6905021039717987]
A ground robot explores an indoor space with a single floor and vertical walls, and collects a sequence of intensity images and 2D LiDAR datasets.
The alignment of sensor outputs and image segmentation are computed jointly by aligning LiDAR points.
The ambiguity in images for ground-wall boundary extraction is removed with the assistance of LiDAR observations.
arXiv Detail & Related papers (2022-04-24T06:26:02Z) - SPIN: Simplifying Polar Invariance for Neural networks Application to
vision-based irradiance forecasting [2.624902795082451]
Unwrapping an image into its polar coordinates provides a more explicit representation to train a convolutional architecture.
We show that this preprocessing step significantly improves prediction results by standardising the scene representation.
This transformation magnifies the area surrounding the centre of the rotation, leading to more accurate short-term irradiance predictions.
arXiv Detail & Related papers (2021-11-29T12:58:57Z) - Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian
Photometric Stereo [61.6260594326246]
We introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously.
Using separable 4D convolutions and 2D heat-maps reduces the size and makes more efficient.
arXiv Detail & Related papers (2021-03-22T18:06:58Z) - Deep Photometric Stereo for Non-Lambertian Surfaces [89.05501463107673]
We introduce a fully convolutional deep network for calibrated photometric stereo, which we call PS-FCN.
PS-FCN learns the mapping from reflectance observations to surface normal, and is able to handle surfaces with general and unknown isotropic reflectance.
To deal with the uncalibrated scenario where light directions are unknown, we introduce a new convolutional network, named LCNet, to estimate light directions from input images.
arXiv Detail & Related papers (2020-07-26T15:20:53Z) - Where am I looking at? Joint Location and Orientation Estimation by
Cross-View Matching [95.64702426906466]
Cross-view geo-localization is a problem given a large-scale database of geo-tagged aerial images.
Knowing orientation between ground and aerial images can significantly reduce matching ambiguity between these two views.
We design a Dynamic Similarity Matching network to estimate cross-view orientation alignment during localization.
arXiv Detail & Related papers (2020-05-08T05:21:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.