CLRNet: Targetless Extrinsic Calibration for Camera, Lidar and 4D Radar Using Deep Learning

• URL: http://arxiv.org/abs/2603.15767v1
• Date: Mon, 16 Mar 2026 18:01:13 GMT
• Title: CLRNet: Targetless Extrinsic Calibration for Camera, Lidar and 4D Radar Using Deep Learning
• Abstract summary: We propose CLRNet, a novel, multi-modal end-to-end deep learning (DL) calibration network.<n>We demonstrate superior calibration accuracy compared to existing state-of-the-art methods.
• Score: 4.316506818580031
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we address extrinsic calibration for camera, lidar, and 4D radar sensors. Accurate extrinsic calibration of radar remains a challenge due to the sparsity of its data. We propose CLRNet, a novel, multi-modal end-to-end deep learning (DL) calibration network capable of addressing joint camera-lidar-radar calibration, or pairwise calibration between any two of these sensors. We incorporate equirectangular projection, camera-based depth image prediction, additional radar channels, and leverage lidar with a shared feature space and loop closure loss. In extensive experiments using the View-of-Delft and Dual-Radar datasets, we demonstrate superior calibration accuracy compared to existing state-of-the-art methods, reducing both median translational and rotational calibration errors by at least 50%. Finally, we examine the domain transfer capabilities of the proposed network and baselines, when evaluating across datasets. The code will be made publicly available upon acceptance at: https://github.com/tudelft-iv.

Related papers

• RbFT-Net: Rectify-Before-Fuse Temporal Radar Anchors for 4D Radar-Camera Depth Completion [20.340336916195053]
  Radar measurements are inherently sparse and susceptible to clutter, multipath reflections, and projection errors.<n>We propose RbFT-Net, an end-to-end framework for multi-frame 4D radar-camera depth completion.<n> Experiments on ZJU-4DRadarCam and a newly collected 4D radar-camera-LiDAR dataset show that RbFT-Net consistently outperforms independent radar-camera methods.
  arXiv  Detail & Related papers  (2026-08-13T11:26:46Z)
• RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields [49.41438378775056]
  Radars are an ideal complement to cameras, with cameras offering fine angular resolution and radars providing metric depth and robustness under adverse weather.<n>RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.
  arXiv  Detail & Related papers  (2026-05-25T21:03:40Z)
• 4D-CAAL: 4D Radar-Camera Calibration and Auto-Labeling for Autonomous Driving [28.84154696792842]
  We propose 4D-CAAL, a unified framework for 4D radar-camera calibration and auto-labeling.<n>We develop a robust correspondence matching algorithm that aligns the checkerboard center with the strongest radar reflection point.<n>We present an auto-labeling pipeline that leverages the calibrated sensor relationship to transfer annotations from camera-based segmentations to radar point clouds.
  arXiv  Detail & Related papers  (2026-01-29T09:30:41Z)
• UniCalib: Targetless LiDAR-Camera Calibration via Probabilistic Flow on Unified Depth Representations [30.56092814783138]
  DF-Calib is a LiDAR-camera calibration method that reformulates calibration as an intra-modality depth flow estimation problem.<n> DF-Calib estimates a dense depth map from the camera image and completes the sparse LiDAR projected depth map.<n>We introduce a reliability map to prioritize valid pixels and propose a perceptually weighted sparse flow loss to enhance depth flow estimation.
  arXiv  Detail & Related papers  (2025-04-02T07:09:44Z)
• RSAR: Restricted State Angle Resolver and Rotated SAR Benchmark [61.987291551925516]
  We introduce the Unit Cycle Resolver, which incorporates a unit circle constraint loss to improve angle prediction accuracy.<n>Our approach can effectively improve the performance of existing state-of-the-art weakly supervised methods.<n>With the aid of UCR, we further annotate and introduce RSAR, the largest multi-class rotated SAR object detection dataset to date.
  arXiv  Detail & Related papers  (2025-01-08T11:41:47Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• Continuous Online Extrinsic Calibration of Fisheye Camera and LiDAR [7.906477322731106]
  An accurate extrinsic calibration is required to fuse the camera and LiDAR data into a common spatial reference frame required by high-level perception functions. There is a need for continuous online extrinsic calibration algorithms which can automatically update the value of the camera-LiDAR calibration during the life of the vehicle using only sensor data. We propose using mutual information between the camera image's depth estimate, provided by commonly available monocular depth estimation networks, and the LiDAR pointcloud's geometric distance as a optimization metric for extrinsic calibration.
  arXiv  Detail & Related papers  (2023-06-22T23:16:31Z)
• Real-time multiview data fusion for object tracking with RGBD sensors [4.68299658663016]
  This paper presents a new approach to accurately track a moving vehicle with a multiview setup of red-green-blue depth (RGBD) cameras. We first propose a correction method to eliminate a shift, which occurs in depth sensors when they become worn. Next, we present a sensor-wise filtering system to correct for an unknown vehicle motion.
  arXiv  Detail & Related papers  (2021-10-28T09:24:49Z)
• Unsupervised Depth Completion with Calibrated Backprojection Layers [79.35651668390496]
  We propose a deep neural network architecture to infer dense depth from an image and a sparse point cloud. It is trained using a video stream and corresponding synchronized sparse point cloud, as obtained from a LIDAR or other range sensor, along with the intrinsic calibration parameters of the camera. At inference time, the calibration of the camera, which can be different from the one used for training, is fed as an input to the network along with the sparse point cloud and a single image.
  arXiv  Detail & Related papers  (2021-08-24T05:41:59Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• Learning Camera Miscalibration Detection [83.38916296044394]
  This paper focuses on a data-driven approach to learn the detection of miscalibration in vision sensors, specifically RGB cameras. Our contributions include a proposed miscalibration metric for RGB cameras and a novel semi-synthetic dataset generation pipeline based on this metric. By training a deep convolutional neural network, we demonstrate the effectiveness of our pipeline to identify whether a recalibration of the camera's intrinsic parameters is required or not.
  arXiv  Detail & Related papers  (2020-05-24T10:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.