Single-Image Depth Prediction Makes Feature Matching Easier
- URL: http://arxiv.org/abs/2008.09497v1
- Date: Fri, 21 Aug 2020 14:25:36 GMT
- Title: Single-Image Depth Prediction Makes Feature Matching Easier
- Authors: Carl Toft, Daniyar Turmukhambetov, Torsten Sattler, Fredrik Kahl,
Gabriel Brostow
- Abstract summary: We show that CNN-based depths inferred from single RGB images are quite helpful, despite their flaws.
They allow us to pre-warp images and rectify perspective distortions, to significantly enhance SIFT and BRISK features.
- Score: 49.13237284669722
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Good local features improve the robustness of many 3D re-localization and
multi-view reconstruction pipelines. The problem is that viewing angle and
distance severely impact the recognizability of a local feature. Attempts to
improve appearance invariance by choosing better local feature points or by
leveraging outside information, have come with pre-requisites that made some of
them impractical. In this paper, we propose a surprisingly effective
enhancement to local feature extraction, which improves matching. We show that
CNN-based depths inferred from single RGB images are quite helpful, despite
their flaws. They allow us to pre-warp images and rectify perspective
distortions, to significantly enhance SIFT and BRISK features, enabling more
good matches, even when cameras are looking at the same scene but in opposite
directions.
Related papers
- Local Feature Extraction from Salient Regions by Feature Map
Transformation [0.7734726150561086]
We propose a framework that robustly extracts and describes salient local features regardless of changing light and viewpoints.
The framework suppresses illumination variations and encourages structural information to ignore the noise from light.
Our model extracts feature points from salient regions leading to reduced incorrect matches.
arXiv Detail & Related papers (2023-01-25T05:31:20Z) - Shared Coupling-bridge for Weakly Supervised Local Feature Learning [0.7366405857677226]
This paper focuses on promoting the currently popular sparse local feature learning with camera pose supervision.
It proposes a Shared Coupling-bridge scheme with four light-weight yet effective improvements for weakly-supervised local feature learning.
It could often obtain a state-of-the-art performance on classic image matching and visual localization.
arXiv Detail & Related papers (2022-12-14T05:47:52Z) - MeshLoc: Mesh-Based Visual Localization [54.731309449883284]
We explore a more flexible alternative based on dense 3D meshes that does not require features matching between database images to build the scene representation.
Surprisingly competitive results can be obtained when extracting features on renderings of these meshes, without any neural rendering stage.
Our results show that dense 3D model-based representations are a promising alternative to existing representations and point to interesting and challenging directions for future research.
arXiv Detail & Related papers (2022-07-21T21:21:10Z) - CPO: Change Robust Panorama to Point Cloud Localization [20.567452635590946]
We present CPO, a robust algorithm that localizes a 2D panorama with respect to a 3D point cloud of a scene possibly containing changes.
CPO is lightweight and achieves effective localization in all tested scenarios.
arXiv Detail & Related papers (2022-07-12T05:10:32Z) - ReF -- Rotation Equivariant Features for Local Feature Matching [30.459559206664427]
We propose an alternative, complementary approach that centers on inducing bias in the model architecture itself to generate rotation-specific' features.
We demonstrate that this high performance, rotation-specific coverage from the steerable CNNs can be expanded to all rotation angles.
We present a detailed analysis of the performance effects of ensembling, robust estimation, network architecture variations, and the use of rotation priors.
arXiv Detail & Related papers (2022-03-10T07:36:09Z) - Pixel-Perfect Structure-from-Motion with Featuremetric Refinement [96.73365545609191]
We refine two key steps of structure-from-motion by a direct alignment of low-level image information from multiple views.
This significantly improves the accuracy of camera poses and scene geometry for a wide range of keypoint detectors.
Our system easily scales to large image collections, enabling pixel-perfect crowd-sourced localization at scale.
arXiv Detail & Related papers (2021-08-18T17:58:55Z) - PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective
Crop Layers [111.55817466296402]
We introduce Perspective Crop Layers (PCLs) - a form of perspective crop of the region of interest based on the camera geometry.
PCLs deterministically remove the location-dependent perspective effects while leaving end-to-end training and the number of parameters of the underlying neural network.
PCL offers an easy way to improve the accuracy of existing 3D reconstruction networks by making them geometry aware.
arXiv Detail & Related papers (2020-11-27T08:48:43Z) - Multi-View Optimization of Local Feature Geometry [70.18863787469805]
We address the problem of refining the geometry of local image features from multiple views without known scene or camera geometry.
Our proposed method naturally complements the traditional feature extraction and matching paradigm.
We show that our method consistently improves the triangulation and camera localization performance for both hand-crafted and learned local features.
arXiv Detail & Related papers (2020-03-18T17:22:11Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.