Self-Supervised Camera Self-Calibration from Video
- URL: http://arxiv.org/abs/2112.03325v1
- Date: Mon, 6 Dec 2021 19:42:05 GMT
- Title: Self-Supervised Camera Self-Calibration from Video
- Authors: Jiading Fang, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Greg
Shakhnarovich, Adrien Gaidon, Matthew R.Walter
- Abstract summary: We propose a learning algorithm to regress per-sequence calibration parameters using an efficient family of general camera models.
Our procedure achieves self-calibration results with sub-pixel reprojection error, outperforming other learning-based methods.
- Score: 34.35533943247917
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Camera calibration is integral to robotics and computer vision algorithms
that seek to infer geometric properties of the scene from visual input streams.
In practice, calibration is a laborious procedure requiring specialized data
collection and careful tuning. This process must be repeated whenever the
parameters of the camera change, which can be a frequent occurrence for mobile
robots and autonomous vehicles. In contrast, self-supervised depth and
ego-motion estimation approaches can bypass explicit calibration by inferring
per-frame projection models that optimize a view synthesis objective. In this
paper, we extend this approach to explicitly calibrate a wide range of cameras
from raw videos in the wild. We propose a learning algorithm to regress
per-sequence calibration parameters using an efficient family of general camera
models. Our procedure achieves self-calibration results with sub-pixel
reprojection error, outperforming other learning-based methods. We validate our
approach on a wide variety of camera geometries, including perspective,
fisheye, and catadioptric. Finally, we show that our approach leads to
improvements in the downstream task of depth estimation, achieving
state-of-the-art results on the EuRoC dataset with greater computational
efficiency than contemporary methods.
Related papers
- Redundancy-Aware Camera Selection for Indoor Scene Neural Rendering [54.468355408388675]
We build a similarity matrix that incorporates both the spatial diversity of the cameras and the semantic variation of the images.
We apply a diversity-based sampling algorithm to optimize the camera selection.
We also develop a new dataset, IndoorTraj, which includes long and complex camera movements captured by humans in virtual indoor environments.
arXiv Detail & Related papers (2024-09-11T08:36:49Z) - VICAN: Very Efficient Calibration Algorithm for Large Camera Networks [49.17165360280794]
We introduce a novel methodology that extends Pose Graph Optimization techniques.
We consider the bipartite graph encompassing cameras, object poses evolving dynamically, and camera-object relative transformations at each time step.
Our framework retains compatibility with traditional PGO solvers, but its efficacy benefits from a custom-tailored optimization scheme.
arXiv Detail & Related papers (2024-03-25T17:47:03Z) - Single-image camera calibration with model-free distortion correction [0.0]
This paper proposes a method for estimating the complete set of calibration parameters from a single image of a planar speckle pattern covering the entire sensor.
The correspondence between image points and physical points on the calibration target is obtained using Digital Image Correlation.
At the end of the procedure, a dense and uniform model-free distortion map is obtained over the entire image.
arXiv Detail & Related papers (2024-03-02T16:51:35Z) - Learning Robust Multi-Scale Representation for Neural Radiance Fields
from Unposed Images [65.41966114373373]
We present an improved solution to the neural image-based rendering problem in computer vision.
The proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time.
arXiv Detail & Related papers (2023-11-08T08:18:23Z) - Deep Learning for Camera Calibration and Beyond: A Survey [100.75060862015945]
Camera calibration involves estimating camera parameters to infer geometric features from captured sequences.
Recent efforts show that learning-based solutions have the potential to be used in place of the repeatability works of manual calibrations.
arXiv Detail & Related papers (2023-03-19T04:00:05Z) - Multi-task Learning for Camera Calibration [3.274290296343038]
We present a unique method for predicting intrinsic (principal point offset and focal length) and extrinsic (baseline, pitch, and translation) properties from a pair of images.
By reconstructing the 3D points using a camera model neural network and then using the loss in reconstruction to obtain the camera specifications, this innovative camera projection loss (CPL) method allows us that the desired parameters should be estimated.
arXiv Detail & Related papers (2022-11-22T17:39:31Z) - Rethinking Generic Camera Models for Deep Single Image Camera
Calibration to Recover Rotation and Fisheye Distortion [8.877834897951578]
We propose a generic camera model that has the potential to address various types of distortion.
Our proposed method outperformed conventional methods on two largescale datasets and images captured by off-the-shelf fisheye cameras.
arXiv Detail & Related papers (2021-11-25T05:58:23Z) - Camera Calibration through Camera Projection Loss [4.36572039512405]
We propose a novel method to predict intrinsic (focal length and principal point offset) parameters using an image pair.
Unlike existing methods, we proposed a new representation that incorporates camera model equations as a neural network in multi-task learning framework.
Our proposed approach achieves better performance with respect to both deep learning-based and traditional methods on 7 out of 10 parameters evaluated.
arXiv Detail & Related papers (2021-10-07T14:03:10Z) - Self-Calibrating Neural Radiance Fields [68.64327335620708]
We jointly learn the geometry of the scene and the accurate camera parameters without any calibration objects.
Our camera model consists of a pinhole model, a fourth order radial distortion, and a generic noise model that can learn arbitrary non-linear camera distortions.
arXiv Detail & Related papers (2021-08-31T13:34:28Z) - Wide-angle Image Rectification: A Survey [86.36118799330802]
wide-angle images contain distortions that violate the assumptions underlying pinhole camera models.
Image rectification, which aims to correct these distortions, can solve these problems.
We present a detailed description and discussion of the camera models used in different approaches.
Next, we review both traditional geometry-based image rectification methods and deep learning-based methods.
arXiv Detail & Related papers (2020-10-30T17:28:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.