Event-RGB Fusion for Spacecraft Pose Estimation Under Harsh Lighting

• URL: http://arxiv.org/abs/2507.05698v1
• Date: Tue, 08 Jul 2025 06:11:42 GMT
• Title: Event-RGB Fusion for Spacecraft Pose Estimation Under Harsh Lighting
• Authors: Mohsi Jawaid, Marcus Märtens, Tat-Jun Chin,
• Abstract summary: Spacecraft pose estimation is crucial for autonomous in-space operations, such as rendezvous, docking and on-orbit servicing.<n> Vision-based pose estimation methods, which typically employ RGB imaging sensors, are challenged by harsh lighting conditions.<n>This work introduces a sensor fusion approach combining RGB and event sensors.
• Score: 20.59391413816475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spacecraft pose estimation is crucial for autonomous in-space operations, such as rendezvous, docking and on-orbit servicing. Vision-based pose estimation methods, which typically employ RGB imaging sensors, is a compelling solution for spacecraft pose estimation, but are challenged by harsh lighting conditions, which produce imaging artifacts such as glare, over-exposure, blooming and lens flare. Due to their much higher dynamic range, neuromorphic or event sensors are more resilient to extreme lighting conditions. However, event sensors generally have lower spatial resolution and suffer from reduced signal-to-noise ratio during periods of low relative motion. This work addresses these individual sensor limitations by introducing a sensor fusion approach combining RGB and event sensors. A beam-splitter prism was employed to achieve precise optical and temporal alignment. Then, a RANSAC-based technique was developed to fuse the information from the RGB and event channels to achieve pose estimation that leveraged the strengths of the two modalities. The pipeline was complemented by dropout uncertainty estimation to detect extreme conditions that affect either channel. To benchmark the performance of the proposed event-RGB fusion method, we collected a comprehensive real dataset of RGB and event data for satellite pose estimation in a laboratory setting under a variety of challenging illumination conditions. Encouraging results on the dataset demonstrate the efficacy of our event-RGB fusion approach and further supports the usage of event sensors for spacecraft pose estimation. To support community research on this topic, our dataset will be released publicly.

Related papers

• Event-Driven Dynamic Scene Depth Completion [50.01494043834177]
  EventDC is the first event-driven depth completion framework.<n>It consists of two key components: Event-Modulated Alignment (EMA) and Local Depth Filtering (LDF)
  arXiv  Detail & Related papers  (2025-05-19T16:02:37Z)
• RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions [0.3141085922386211]
  Short-wave infrared (SWIR) imaging offers several advantages over NIR and LWIR.<n>Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions.<n>We introduce the RGB and SWIR Multispectral Driving dataset, which comprises 100,000 synchronized and spatially aligned RGB-SWIR image pairs.
  arXiv  Detail & Related papers  (2025-04-10T09:54:57Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera [8.673063170884591]
  EOLO is a novel object detection framework that achieves robust and efficient all-day detection by fusing both RGB and event modalities. Our EOLO framework is built based on a lightweight spiking neural network (SNN) to efficiently leverage the asynchronous property of events.
  arXiv  Detail & Related papers  (2023-09-17T15:14:01Z)
• Deformable Neural Radiance Fields using RGB and Event Cameras [65.40527279809474]
  We develop a novel method to model the deformable neural radiance fields using RGB and event cameras. The proposed method uses the asynchronous stream of events and sparse RGB frames. Experiments conducted on both realistically rendered graphics and real-world datasets demonstrate a significant benefit of the proposed method.
  arXiv  Detail & Related papers  (2023-09-15T14:19:36Z)
• Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor [58.305341034419136]
  We present the first dense SLAM system with a monocular camera and a light-weight ToF sensor. We propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor. Experiments demonstrate that our system well exploits the signals of light-weight ToF sensors and achieves competitive results.
  arXiv  Detail & Related papers  (2023-08-28T07:56:13Z)
• Attentive Multimodal Fusion for Optical and Scene Flow [24.08052492109655]
  Existing methods typically rely solely on RGB images or fuse the modalities at later stages. We propose a novel deep neural network approach named FusionRAFT, which enables early-stage information fusion between sensor modalities. Our approach exhibits improved robustness in the presence of noise and low-lighting conditions that affect the RGB images.
  arXiv  Detail & Related papers  (2023-07-28T04:36:07Z)
• EVEN: An Event-Based Framework for Monocular Depth Estimation at Adverse Night Conditions [14.390463371184566]
  We study monocular depth estimation at night time in which various adverse weather, light, and different road conditions exist. We propose an event-vision based framework that integrates low-light enhancement for the RGB source, and exploits the merits of RGB and event data.
  arXiv  Detail & Related papers  (2023-02-08T03:35:47Z)
• Learning Enriched Illuminants for Cross and Single Sensor Color Constancy [182.4997117953705]
  We propose cross-sensor self-supervised training to train the network. We train the network by randomly sampling the artificial illuminants in a sensor-independent manner. Experiments show that our cross-sensor model and single-sensor model outperform other state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-03-21T15:45:35Z)
• Multi-Modal Fusion for Sensorimotor Coordination in Steering Angle Prediction [8.707695512525717]
  Imitation learning is employed to learn sensorimotor coordination for steering angle prediction in an end-to-end fashion. This work explores the fusion of frame-based RGB and event data for learning end-to-end lateral control. We propose DRFuser, a novel convolutional encoder-decoder architecture for learning end-to-end lateral control.
  arXiv  Detail & Related papers  (2022-02-11T08:22:36Z)
• Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning [59.19469551774703]
  Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image. We construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle. Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parking lots, and other scenarios from day to night.
  arXiv  Detail & Related papers  (2020-03-05T05:29:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.