Image-to-Point Cloud Registration Made Easy with Rectified Flow-based LiDAR Upsampling

• URL: http://arxiv.org/abs/2607.14639v1
• Date: Thu, 16 Jul 2026 07:06:04 GMT
• Title: Image-to-Point Cloud Registration Made Easy with Rectified Flow-based LiDAR Upsampling
• Abstract summary: Image-to-Point Cloud Registration (I2P) is essential for integrating camera and LiDAR in perception and autonomous systems.<n>We propose a simple yet effective I2P method that treats LiDAR as an imaging sensor from a single sparse LiDAR scan.<n> Experiments on the R3LIVE dataset show that the proposed method achieves a mean error of 4.89 / 1.63 m, outperforming existing methods.
• Score: 15.063950064670403
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-to-Point Cloud Registration (I2P) is essential for integrating camera and LiDAR in perception and autonomous systems, yet the modality gap between images and point clouds makes it difficult to achieve both high accuracy and strong generalization. In this paper, we propose a simple yet effective I2P method that treats LiDAR as an imaging sensor: from a single sparse LiDAR scan, we generate a dense LiDAR intensity image using Conditional Rectified Flow, match it with a camera image using a pre-trained feature matcher, and estimate the 6-DoF relative pose via PnP-RANSAC. The proposed model is pre-trained through a self-supervised image completion task and fine-tuned on a small amount of LiDAR data (neither image-point cloud pairs nor ground-truth sensor poses are required), enabling it to scale to diverse LiDAR and camera configurations. Experiments on the R3LIVE dataset show that the proposed method achieves a mean error of 4.89° / 1.63 m, outperforming existing methods, while completing a single registration in approximately 0.68 s.

Related papers

• DPA-I2P: Depth-Guided Projective Alignment for Image-to-Point-Cloud Registration in Autonomous Driving [12.021166543643941]
  Image-to-Point Cloud Registration aims to estimate the camera pose of a given image within a 3D scene point cloud.<n>Recent implicit correspondence learning methods have improved registration performance by learning cross-modal alignment.<n>We propose Depth-Guided Projective Alignment for Image-to-Point-Cloud Registration (DPA-I2P)
  arXiv  Detail & Related papers  (2026-08-27T04:01:15Z)
• A Height-Constrained 2-Point Minimal Solver for Pose Estimation from Active LED Markers with Event Cameras [51.7262911415172]
  We develop an event-based active marker system to collect real-world data with ground truth from a motion capture system.<n>Experiments on both synthetic and real data demonstrate improved accuracy over the state-of-the-art P2P solver.
  arXiv  Detail & Related papers  (2026-08-10T12:19:24Z)
• CLAIM: Camera-LiDAR Alignment with Intensity and Monodepth [10.382931594335451]
  We propose CLAIM, a novel method of aligning data from the camera and LiDAR.<n>Given the initial guess and pairs of images and LiDAR point clouds, CLAIM utilizes a coarse-to-fine searching method to find the optimal transformation.<n>We validate CLAIM on public KITTI, datasets, and MIAS-L CEC, and the experimental results demonstrate its superior performance compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-12-16T01:46:24Z)
• Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching [7.5461100059974315]
  We introduce a detector-free framework for direct point-pixel matching between LiDAR and camera views.<n>Specifically, we project the LiDAR intensity map into a 2D view from the LiDAR perspective and feed it into an attention-based matching network.<n>To further enhance matching reliability, we introduce a repeatability scoring mechanism that acts as a soft visibility prior.
  arXiv  Detail & Related papers  (2025-06-28T06:57:13Z)
• SuperFlow++: Enhanced Spatiotemporal Consistency for Cross-Modal Data Pretraining [62.433137130087445]
  SuperFlow++ is a novel framework that integrates pretraining and downstream tasks using consecutive camera pairs.<n>We show that SuperFlow++ outperforms state-of-the-art methods across diverse tasks and driving conditions.<n>With strong generalizability and computational efficiency, SuperFlow++ establishes a new benchmark for data-efficient LiDAR-based perception in autonomous driving.
  arXiv  Detail & Related papers  (2025-03-25T17:59:57Z)
• EEPNet-V2: Patch-to-Pixel Solution for Efficient Cross-Modal Registration between LiDAR Point Cloud and Camera Image [15.975048012603914]
  We propose a framework that projects point clouds into several 2D representations for matching with camera images.<n>To tackle the challenges of cross modal differences and the limited overlap between LiDAR point clouds and images in the image matching task, we introduce a multi-scale feature extraction network.<n>We validate the performance of our model through experiments on the KITTI and nuScenes datasets.
  arXiv  Detail & Related papers  (2025-03-19T15:04:01Z)
• LiDAR-GS:Real-time LiDAR Re-Simulation using Gaussian Splatting [53.58528891081709]
  We present LiDAR-GS, a real-time, high-fidelity re-simulation of LiDAR scans in public urban road scenes.<n>The method achieves state-of-the-art results in both rendering frame rate and quality on publically available large scene datasets.
  arXiv  Detail & Related papers  (2024-10-07T15:07:56Z)
• 4D Contrastive Superflows are Dense 3D Representation Learners [62.433137130087445]
  We introduce SuperFlow, a novel framework designed to harness consecutive LiDAR-camera pairs for establishing pretraining objectives. To further boost learning efficiency, we incorporate a plug-and-play view consistency module that enhances alignment of the knowledge distilled from camera views.
  arXiv  Detail & Related papers  (2024-07-08T17:59:54Z)
• UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation [51.443788294845845]
  We present UltraLiDAR, a data-driven framework for scene-level LiDAR completion, LiDAR generation, and LiDAR manipulation. We show that by aligning the representation of a sparse point cloud to that of a dense point cloud, we can densify the sparse point clouds. By learning a prior over the discrete codebook, we can generate diverse, realistic LiDAR point clouds for self-driving.
  arXiv  Detail & Related papers  (2023-11-02T17:57:03Z)
• Gait Recognition in Large-scale Free Environment via Single LiDAR [35.684257181154905]
  LiDAR's ability to capture depth makes it pivotal for robotic perception and holds promise for real-world gait recognition. We present the Hierarchical Multi-representation Feature Interaction Network (HMRNet) for robust gait recognition. To facilitate LiDAR-based gait recognition research, we introduce FreeGait, a comprehensive gait dataset from large-scale, unconstrained settings.
  arXiv  Detail & Related papers  (2022-11-22T16:05:58Z)
• Boosting 3D Object Detection by Simulating Multimodality on Point Clouds [51.87740119160152]
  This paper presents a new approach to boost a single-modality (LiDAR) 3D object detector by teaching it to simulate features and responses that follow a multi-modality (LiDAR-image) detector. The approach needs LiDAR-image data only when training the single-modality detector, and once well-trained, it only needs LiDAR data at inference. Experimental results on the nuScenes dataset show that our approach outperforms all SOTA LiDAR-only 3D detectors.
  arXiv  Detail & Related papers  (2022-06-30T01:44:30Z)
• LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation [78.74202673902303]
  We propose a coarse-tofine LiDAR and camera fusion-based network (termed as LIF-Seg) for LiDAR segmentation. The proposed method fully utilizes the contextual information of images and introduces a simple but effective early-fusion strategy. The cooperation of these two components leads to the success of the effective camera-LiDAR fusion.
  arXiv  Detail & Related papers  (2021-08-17T08:53:11Z)
• A Simple and Efficient Registration of 3D Point Cloud and Image Data for Indoor Mobile Mapping System [18.644879251473647]
  registration of 3D LiDAR point clouds with optical images is critical in the combination of multi-source data. Geometric misalignment originally exists in the pose data between LiDAR point clouds and optical images. We develop a simple but efficient registration method to improve the accuracy of the initial pose.
  arXiv  Detail & Related papers  (2020-10-27T13:01:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.