UniCal: Unified Neural Sensor Calibration

• URL: http://arxiv.org/abs/2409.18953v1
• Date: Fri, 27 Sep 2024 17:56:04 GMT
• Title: UniCal: Unified Neural Sensor Calibration
• Authors: Ze Yang, George Chen, Haowei Zhang, Kevin Ta, Ioan Andrei Bârsan, Daniel Murphy, Sivabalan Manivasagam, Raquel Urtasun,
• Abstract summary: Self-driving vehicles (SDVs) require accurate calibration of LiDARs and cameras to fuse sensor data accurately for autonomy. Traditional calibration methods leverage fiducials captured in a controlled and structured scene and compute correspondences to optimize over. We propose UniCal, a unified framework for effortlessly calibrating SDVs equipped with multiple LiDARs and cameras.
• Score: 32.7372115947273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-driving vehicles (SDVs) require accurate calibration of LiDARs and cameras to fuse sensor data accurately for autonomy. Traditional calibration methods typically leverage fiducials captured in a controlled and structured scene and compute correspondences to optimize over. These approaches are costly and require substantial infrastructure and operations, making it challenging to scale for vehicle fleets. In this work, we propose UniCal, a unified framework for effortlessly calibrating SDVs equipped with multiple LiDARs and cameras. Our approach is built upon a differentiable scene representation capable of rendering multi-view geometrically and photometrically consistent sensor observations. We jointly learn the sensor calibration and the underlying scene representation through differentiable volume rendering, utilizing outdoor sensor data without the need for specific calibration fiducials. This "drive-and-calibrate" approach significantly reduces costs and operational overhead compared to existing calibration systems, enabling efficient calibration for large SDV fleets at scale. To ensure geometric consistency across observations from different sensors, we introduce a novel surface alignment loss that combines feature-based registration with neural rendering. Comprehensive evaluations on multiple datasets demonstrate that UniCal outperforms or matches the accuracy of existing calibration approaches while being more efficient, demonstrating the value of UniCal for scalable calibration.

Related papers

• Kalib: Markerless Hand-Eye Calibration with Keypoint Tracking [52.4190876409222]
  Hand-eye calibration involves estimating the transformation between the camera and the robot. Recent advancements in deep learning offer markerless techniques, but they present challenges. We propose Kalib, an automatic and universal markerless hand-eye calibration pipeline.
  arXiv  Detail & Related papers  (2024-08-20T06:03:40Z)
• SOAC: Spatio-Temporal Overlap-Aware Multi-Sensor Calibration using Neural Radiance Fields [10.958143040692141]
  In rapidly-evolving domains such as autonomous driving, the use of multiple sensors with different modalities is crucial to ensure operational precision and stability. To correctly exploit the provided information by each sensor in a single common frame, it is essential for these sensors to be accurately calibrated. We leverage the ability of Neural Radiance Fields to represent different modalities in a common representation.
  arXiv  Detail & Related papers  (2023-11-27T13:25:47Z)
• CalibFormer: A Transformer-based Automatic LiDAR-Camera Calibration Network [11.602943913324653]
  CalibFormer is an end-to-end network for automatic LiDAR-camera calibration. We aggregate multiple layers of camera and LiDAR image features to achieve high-resolution representations. Our method achieved a mean translation error of $0.8751 mathrmcm$ and a mean rotation error of $0.0562 circ$ on the KITTI dataset.
  arXiv  Detail & Related papers  (2023-11-26T08:59:30Z)
• From Chaos to Calibration: A Geometric Mutual Information Approach to Target-Free Camera LiDAR Extrinsic Calibration [4.378156825150505]
  We propose a target free extrinsic calibration algorithm that requires no ground truth training data. We demonstrate our proposed improvement using the KITTI and KITTI-360 fisheye data set.
  arXiv  Detail & Related papers  (2023-11-03T13:30:31Z)
• MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration [4.405687114738899]
  We take advantage of recent advances in computer graphics and implicit volumetric scene representation to tackle the problem of multi-sensor spatial and temporal calibration. Our method enables accurate and robust calibration from data captured in uncontrolled and unstructured urban environments. We demonstrate the accuracy and robustness of our method in urban scenes typically encountered in autonomous driving scenarios.
  arXiv  Detail & Related papers  (2023-03-06T11:59:13Z)
• TrajMatch: Towards Automatic Spatio-temporal Calibration for Roadside LiDARs through Trajectory Matching [12.980324010888664]
  We propose TrajMatch -- the first system that can automatically calibrate for roadside LiDARs in both time and space. Experiment results show that TrajMatch can achieve a spatial calibration error of less than 10cm and a temporal calibration error of less than 1.5ms.
  arXiv  Detail & Related papers  (2023-02-04T12:27:01Z)
• How to Calibrate Your Event Camera [58.80418612800161]
  We propose a generic event camera calibration framework using image reconstruction. We show that neural-network-based image reconstruction is well suited for the task of intrinsic and extrinsic calibration of event cameras.
  arXiv  Detail & Related papers  (2021-05-26T07:06:58Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• Learning Camera Miscalibration Detection [83.38916296044394]
  This paper focuses on a data-driven approach to learn the detection of miscalibration in vision sensors, specifically RGB cameras. Our contributions include a proposed miscalibration metric for RGB cameras and a novel semi-synthetic dataset generation pipeline based on this metric. By training a deep convolutional neural network, we demonstrate the effectiveness of our pipeline to identify whether a recalibration of the camera's intrinsic parameters is required or not.
  arXiv  Detail & Related papers  (2020-05-24T10:32:49Z)
• Deep Soft Procrustes for Markerless Volumetric Sensor Alignment [81.13055566952221]
  In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
  arXiv  Detail & Related papers  (2020-03-23T10:51:32Z)
• Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
  Miscalibration is a mismatch between a model's confidence and its correctness. We show that focal loss allows us to learn models that are already very well calibrated. We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
  arXiv  Detail & Related papers  (2020-02-21T17:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.