Targetless LiDAR-Camera Calibration with Neural Gaussian Splatting

• URL: http://arxiv.org/abs/2504.04597v2
• Date: Thu, 09 Oct 2025 03:55:36 GMT
• Title: Targetless LiDAR-Camera Calibration with Neural Gaussian Splatting
• Authors: Haebeom Jung, Namtae Kim, Jungwoo Kim, Jaesik Park,
• Abstract summary: We present a Targetless LiDAR scene (TLC) that jointly optimize sensor poses with a neural-based Gaussian representation.<n>Our fully differentiable pipeline with photometric and geometric regularization achieves robust and generalizable calibration.
• Score: 28.41267004945046
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate LiDAR-camera calibration is crucial for multi-sensor systems. However, traditional methods often rely on physical targets, which are impractical for real-world deployment. Moreover, even carefully calibrated extrinsics can degrade over time due to sensor drift or external disturbances, necessitating periodic recalibration. To address these challenges, we present a Targetless LiDAR-Camera Calibration (TLC-Calib) that jointly optimizes sensor poses with a neural Gaussian-based scene representation. Reliable LiDAR points are frozen as anchor Gaussians to preserve global structure, while auxiliary Gaussians prevent local overfitting under noisy initialization. Our fully differentiable pipeline with photometric and geometric regularization achieves robust and generalizable calibration, consistently outperforming existing targetless methods on KITTI-360, Waymo, and FAST-LIVO2, and surpassing even the provided calibrations in rendering quality.

Related papers

• DST-Calib: A Dual-Path, Self-Supervised, Target-Free LiDAR-Camera Extrinsic Calibration Network [57.22935789233992]
  This article presents the first self-supervised LiDAR-camera extrinsic calibration network that operates in an online fashion.<n>The proposed method significantly outperforms existing approaches in terms of generalizability.
  arXiv  Detail & Related papers  (2026-01-03T13:57:01Z)
• ALICE-LRI: A General Method for Lossless Range Image Generation for Spinning LiDAR Sensors without Calibration Metadata [0.0]
  3D LiDAR sensors are essential for autonomous navigation, environmental monitoring, and precision mapping in remote sensing applications.<n>To efficiently process the massive point clouds generated by these sensors, LiDAR data is often projected into 2D range images that organize points by their angular positions and distances.<n>We present ALICE-LRI, the first general, sensor-agnostic method that achieves lossless range image generation from spinning LiDAR point clouds without requiring manufacturer metadata or calibration files.
  arXiv  Detail & Related papers  (2025-10-23T16:22:58Z)
• Coca-Splat: Collaborative Optimization for Camera Parameters and 3D Gaussians [26.3996055215988]
  Coca-Splat is a novel approach to address the challenges of sparse view pose-free scene reconstruction and novel view synthesis (NVS)<n>Inspired by deformable DEtection TRansformer, we design separate queries for 3D Gaussians and camera parameters.<n>We update them layer by layer through deformable Transformer layers, enabling joint optimization in a single network.
  arXiv  Detail & Related papers  (2025-04-01T10:48:46Z)
• Robust LiDAR-Camera Calibration with 2D Gaussian Splatting [0.3281128493853064]
  A critical and initial step in integrating the LiDAR and camera data is the calibration of the LiDAR-camera system.<n>Most existing calibration methods rely on auxiliary target objects, which often involve complex manual operations.<n>We propose a calibration method that estimates LiDAR-camera extrinsic parameters using geometric constraints.
  arXiv  Detail & Related papers  (2025-04-01T08:19:26Z)
• D3DR: Lighting-Aware Object Insertion in Gaussian Splatting [48.80431740983095]
  We propose a method, dubbed D3DR, for inserting a 3DGS-parametrized object into 3DGS scenes. We leverage advances in diffusion models, which, trained on real-world data, implicitly understand correct scene lighting. We demonstrate the method's effectiveness by comparing it to existing approaches.
  arXiv  Detail & Related papers  (2025-03-09T19:48:00Z)
• CalibRefine: Deep Learning-Based Online Automatic Targetless LiDAR-Camera Calibration with Iterative and Attention-Driven Post-Refinement [5.069968819561576]
  CalibRefine is a fully automatic, targetless, and online calibration framework.<n>It directly processes raw LiDAR point clouds and camera images.<n>Our results show that robust object-level feature matching, combined with iterative refinement and self-supervised attention-based refinement, enables reliable sensor alignment.
  arXiv  Detail & Related papers  (2025-02-24T20:53:42Z)
• SC-OmniGS: Self-Calibrating Omnidirectional Gaussian Splatting [29.489453234466982]
  SC- OmniGS is a novel self-calibrating system for fast and accurate radiance field reconstruction using 360-degree images.<n>We introduce a differentiable omnidirectional camera model in order to rectify the distortion of real-world data for performance enhancement.
  arXiv  Detail & Related papers  (2025-02-07T08:06:30Z)
• What Really Matters for Learning-based LiDAR-Camera Calibration [50.2608502974106]
  This paper revisits the development of learning-based LiDAR-Camera calibration.<n>We identify the critical limitations of regression-based methods with the widely used data generation pipeline.<n>We also investigate how the input data format and preprocessing operations impact network performance.
  arXiv  Detail & Related papers  (2025-01-28T14:12:32Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• Improving Gaussian Splatting with Localized Points Management [52.009874685460694]
  Localized Point Management (LPM) is capable of identifying those error-contributing zones in greatest need for both point addition and geometry calibration.<n>LPM applies point densification in the identified zones and then reset the opacity of the points in front of these regions, creating a new opportunity to correct poorly conditioned points.<n> Notably, LPM improves both static 3DGS and dynamic SpaceTimeGS to achieve state-of-the-art rendering quality while retaining real-time speeds.
  arXiv  Detail & Related papers  (2024-06-06T16:55:07Z)
• W-HMR: Monocular Human Mesh Recovery in World Space with Weak-Supervised Calibration [57.37135310143126]
  Previous methods for 3D motion recovery from monocular images often fall short due to reliance on camera coordinates. We introduce W-HMR, a weak-supervised calibration method that predicts "reasonable" focal lengths based on body distortion information. We also present the OrientCorrect module, which corrects body orientation for plausible reconstructions in world space.
  arXiv  Detail & Related papers  (2023-11-29T09:02:07Z)
• P2O-Calib: Camera-LiDAR Calibration Using Point-Pair Spatial Occlusion Relationship [1.6921147361216515]
  We propose a novel target-less calibration approach based on the 2D-3D edge point extraction using the occlusion relationship in 3D space. Our method achieves low error and high robustness that can contribute to the practical applications relying on high-quality Camera-LiDAR calibration.
  arXiv  Detail & Related papers  (2023-11-04T14:32:55Z)
• Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction [54.00007868515432]
  Existing methods face challenges in estimating the accurate correction field due to the uniform velocity assumption. We propose a geometry-based Quadratic Rolling Shutter (QRS) motion solver, which precisely estimates the high-order correction field of individual pixels. Our method surpasses the state-of-the-art by +4.98, +0.77, and +4.33 of PSNR on Carla-RS, Fastec-RS, and BS-RSC datasets, respectively.
  arXiv  Detail & Related papers  (2023-03-31T15:09:18Z)
• Photometric LiDAR and RGB-D Bundle Adjustment [3.3948742816399697]
  This paper presents a novel Bundle Adjustment (BA) photometric strategy that accounts for both RGB-D and LiDAR in the same way. In addition, we present the benefit of jointly using RGB-D and LiDAR within our unified method.
  arXiv  Detail & Related papers  (2023-03-29T17:35:23Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• How to Calibrate Your Event Camera [58.80418612800161]
  We propose a generic event camera calibration framework using image reconstruction. We show that neural-network-based image reconstruction is well suited for the task of intrinsic and extrinsic calibration of event cameras.
  arXiv  Detail & Related papers  (2021-05-26T07:06:58Z)
• CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes [16.201111055979453]
  We propose a novel method to calibrate the extrinsic parameter for LiDAR and camera in road scenes. Our method introduces line features from static straight-line-shaped objects such as road lanes and poles in both image and point cloud. We conduct extensive experiments on KITTI and our in-house dataset, quantitative and qualitative results demonstrate the robustness and accuracy of our method.
  arXiv  Detail & Related papers  (2021-03-08T06:02:44Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems [11.787271829250805]
  Solid-State LiDAR (SSL) enables low-cost and efficient obtainment of 3D point clouds from the environment. We propose a fully automatic calibration method for the non-repetitive scanning SSL and camera systems. We evaluate the proposed method on different types of LiDAR and camera sensor combinations in real conditions.
  arXiv  Detail & Related papers  (2020-11-17T09:11:28Z)
• Learning Camera Miscalibration Detection [83.38916296044394]
  This paper focuses on a data-driven approach to learn the detection of miscalibration in vision sensors, specifically RGB cameras. Our contributions include a proposed miscalibration metric for RGB cameras and a novel semi-synthetic dataset generation pipeline based on this metric. By training a deep convolutional neural network, we demonstrate the effectiveness of our pipeline to identify whether a recalibration of the camera's intrinsic parameters is required or not.
  arXiv  Detail & Related papers  (2020-05-24T10:32:49Z)
• Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials [65.95928593628128]
  We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique. Our algorithm is suitable for perspective cameras and nearby point light sources.
  arXiv  Detail & Related papers  (2020-01-18T12:26:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.