Cyclops: LiDAR as a Camera That Dreams in Color

• URL: http://arxiv.org/abs/2608.16264v1
• Date: Mon, 17 Aug 2026 08:37:09 GMT
• Title: Cyclops: LiDAR as a Camera That Dreams in Color
• Abstract summary: We propose a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks.<n>Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module.<n>The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching.
• Score: 32.25721672776894
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conventionally, robotic perception relies heavily on cameras due to the rich semantic texture they provide. However, their performance degrades significantly in low-light or high-dynamic-range environments. Conversely, while Light Detection and Ranging (LiDAR) captures illumination-invariant geometric and intensity properties, the resulting data are typically single-channel and sparse, creating a significant modality gap when applying vision models pre-trained on RGB datasets. In this paper, we propose Cyclops, a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks. Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module, serving as a geometrically rich source condition. The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching (LBM) with a learned velocity field in a few ODE integration steps. To mitigate inter-frame flickering, we inject prior-frame context via temporal attention layers and further formulate the velocity field as a policy optimized by a differentiable terminal reward that encourages terminal fidelity through backpropagation along the ODE trajectory. Extensive experiments demonstrate that the synthesized RGB, including those generated under near-dark conditions, enable standard RGB-based perception models to substantially outperform both LiDAR baselines and conventional cameras on semantic segmentation, lane detection, and point cloud colorization across diverse lighting conditions.

Related papers

• Event-RGB Adaptive Tracking for Nighttime Highway Perception [7.388485566907394]
  Intelligent Transportation Systems predominantly rely on conventional RGB cameras for traffic perception and vehicle tracking.<n>Nighttime scenarios suffer from motion blur, insufficient exposure, and poor signal-to-noise ratios, which catastrophically impair the reliability of RGB-based sensing systems.<n>We propose a novel Joint Event-RGB Adaptive Tracking (JEAT) framework to address these limitations.
  arXiv  Detail & Related papers  (2026-07-13T14:59:46Z)
• Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing [55.13802890769086]
  We present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing.<n>To achieve profound modal synergy, Evita embeds a suite of intrinsic co-learning modules directly into every encoder layer.<n>Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.
  arXiv  Detail & Related papers  (2026-07-10T06:52:09Z)
• UniRED: Unified RGB-D Video Frame Interpolation with Event Guidance [19.80142530142947]
  High frame-rate RGB-D videos are crucial for a variety of downstream tasks, including motion analysis, dynamic scene understanding, and 3D reconstruction.<n>Existing video methods have achieved strong performance on RGB data, but they are not readily applicable to RGB-D scenarios.<n>We propose a unified multimodal framework for RGB-D video that jointly exploits RGB appearance, depth geometry, and event-based temporal cues.
  arXiv  Detail & Related papers  (2026-06-23T08:03:58Z)
• AdaFuse-Det: Adaptive Cross-Modal Fusion of Event Cameras for Robust Object Detection in Low-Light RGB Imagery [2.804640719143954]
  We present AdaFuse-Det, a dual-stream framework that fuses CLAHE-enhanced RGB frames with voxelized event tensors.<n>On the LLE-VOS benchmark, AdaFuse-Det achieves a Recall of $65.54%$, Precision of $53.85%$, and F1-Score of $59.12%$ under severe illumination degradation.
  arXiv  Detail & Related papers  (2026-05-23T18:01:48Z)
• Spatio-Temporal Difference Guided Motion Deblurring with the Complementary Vision Sensor [12.418360718687026]
  Motion blur arises when rapid scene changes occur during the exposure period, collapsing rich intra-exposure motion into a single RGB frame.<n>We propose Spatio-Temporal Difference Guided Deblur Net (STGDNet) to restore structure and color details lost in blurry RGB inputs.<n>Our method outperforms current RGB or event-based approaches in both synthetic CVS dataset and real-world evaluations.
  arXiv  Detail & Related papers  (2026-04-12T09:54:14Z)
• Spatial Orthogonal Refinement for Robust RGB-Event Visual Object Tracking [8.019596736149834]
  We propose SOR-Track, a framework for robust RGB-Event tracking based on Spatial Orthogonal Refinement (SOR)<n>Experiments on the large-scale FE108 benchmark demonstrate that SOR-Track consistently outperforms existing fusion-based trackers.<n>Despite its simplicity, the proposed method offers a principled and physics-grounded approach to multi-modal feature alignment and texture rectification.
  arXiv  Detail & Related papers  (2026-03-29T23:54:31Z)
• IrisNet: Infrared Image Status Awareness Meta Decoder for Infrared Small Targets Detection [92.56025546608699]
  IrisNet is a novel meta-learned framework that adapts detection strategies to the input infrared image status.<n>Our approach establishes a dynamic mapping between infrared image features and entire decoder parameters.<n> Experiments on NUDT-SIRST, NUAA-SIRST, and IRSTD-1K datasets demonstrate the superiority of our IrisNet.
  arXiv  Detail & Related papers  (2025-11-25T13:53:54Z)
• Dark-EvGS: Event Camera as an Eye for Radiance Field in the Dark [51.68144172958247]
  We propose Dark-EvGS, the first event-assisted 3D GS framework that enables the reconstruction of bright frames from arbitrary viewpoints.<n>Our method achieves better results than existing methods, conquering radiance field reconstruction under challenging low-light conditions.
  arXiv  Detail & Related papers  (2025-07-16T05:54:33Z)
• LiDAR-GS:Real-time LiDAR Re-Simulation using Gaussian Splatting [53.58528891081709]
  We present LiDAR-GS, a real-time, high-fidelity re-simulation of LiDAR scans in public urban road scenes.<n>The method achieves state-of-the-art results in both rendering frame rate and quality on publically available large scene datasets.
  arXiv  Detail & Related papers  (2024-10-07T15:07:56Z)
• Revisiting Event-based Video Frame Interpolation [49.27404719898305]
  Dynamic vision sensors or event cameras provide rich complementary information for video frame. estimating optical flow from events is arguably more difficult than from RGB information. We propose a divide-and-conquer strategy in which event-based intermediate frame synthesis happens incrementally in multiple simplified stages.
  arXiv  Detail & Related papers  (2023-07-24T06:51:07Z)
• Spatiotemporally Consistent HDR Indoor Lighting Estimation [66.26786775252592]
  We propose a physically-motivated deep learning framework to solve the indoor lighting estimation problem. Given a single LDR image with a depth map, our method predicts spatially consistent lighting at any given image position. Our framework achieves photorealistic lighting prediction with higher quality compared to state-of-the-art single-image or video-based methods.
  arXiv  Detail & Related papers  (2023-05-07T20:36:29Z)
• Self-Aligning Depth-regularized Radiance Fields for Asynchronous RGB-D Sequences [12.799443250845224]
  We propose a novel time-pose function, which is an implicit network that maps timestamps to $rm SE(3)$ elements. Our algorithm consists of three steps: (1) time-pose function fitting, (2) radiance field bootstrapping, (3) joint pose error compensation and radiance field refinement. We also show qualitatively improved results on a real-world asynchronous RGB-D sequence captured by drone.
  arXiv  Detail & Related papers  (2022-11-14T15:37:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.