eNavi: Event-based Imitation Policies for Low-Light Indoor Mobile Robot Navigation

• URL: http://arxiv.org/abs/2603.14397v1
• Date: Sun, 15 Mar 2026 14:25:40 GMT
• Title: eNavi: Event-based Imitation Policies for Low-Light Indoor Mobile Robot Navigation
• Abstract summary: We introduce a real-world indoor person-following dataset collected using a TurtleBot 2 robot.<n>We build a multimodal data preprocessing pipeline that temporally aligns event and RGB observations.<n>We propose a late-fusion RGB-Event navigation policy that combines dual MobileNet encoders with a transformer-based fusion module trained via behavioral cloning.
• Score: 22.299468922396894
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Event cameras provide high dynamic range and microsecond-level temporal resolution, making them well-suited for indoor robot navigation, where conventional RGB cameras degrade under fast motion or low-light conditions. Despite advances in event-based perception spanning detection, SLAM, and pose estimation, there remains limited research on end-to-end control policies that exploit the asynchronous nature of event streams. To address this gap, we introduce a real-world indoor person-following dataset collected using a TurtleBot 2 robot, featuring synchronized raw event streams, RGB frames, and expert control actions across multiple indoor maps, trajectories under both normal and low-light conditions. We further build a multimodal data preprocessing pipeline that temporally aligns event and RGB observations while reconstructing ground-truth actions from odometry to support high-quality imitation learning. Building on this dataset, we propose a late-fusion RGB-Event navigation policy that combines dual MobileNet encoders with a transformer-based fusion module trained via behavioral cloning. A systematic evaluation of RGB-only, Event-only, and RGB-Event fusion models across 12 training variations ranging from single-path imitation to general multi-path imitation shows that policies incorporating event data, particularly the fusion model, achieve improved robustness and lower action prediction error, especially in unseen low-light conditions where RGB-only models fail. We release the dataset, synchronization pipeline, and trained models at https://eventbasedvision.github.io/eNavi/

Related papers

• Paths: Prompt-aware Spatio-temporal Transformer with Hierarchical Multi-modal Fusion for RGB-Event Video Person Re-Identification [75.15027201505912]
  We propose a unified framework with disc-temporal modeling and hierarchical multi-modal fusion for RE-VReID.<n>Our framework can learn robust and discriminative representations for RE-VReID.
  arXiv  Detail & Related papers  (2026-08-13T10:58:47Z)
• Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking [58.338848435042536]
  RGB-Event tracking improves localization by fusing RGB appearance textures and dense temporal motion cues from event sensors.<n>Real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion.<n>We present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation.
  arXiv  Detail & Related papers  (2026-06-24T23:34:06Z)
• REALM: An RGB and Event Aligned Latent Manifold for Cross-Modal Perception [20.862898938880306]
  Event cameras offer unique advantages over standard frame-based sensors, including high temporal resolution, low latency, and robustness to extreme lighting.<n>We present REALM, a cross-modal framework that learns an RGB and Event Aligned Latent Manifold.<n>We demonstrate state-of-the-art performance in wide-baseline feature matching, significantly outperforming specialized architectures.
  arXiv  Detail & Related papers  (2026-04-30T22:14:36Z)
• PFM-VEPAR: Prompting Foundation Models for RGB-Event Camera based Pedestrian Attribute Recognition [8.09519291579794]
  Event-based pedestrian recognition (PAR) leverages motion cues to enhance RGB cameras in low-light and motion-blur scenarios.<n>This paper proposes an Event Prompter to apply Discrete Cosine Transform (DCT) and Inverse DCT (IDCT) operations to the event data.<n>An external memory bank designed to provide rich prior knowledge, combined with modern Hopfield networks, enables associative memory-augmented representation learning.
  arXiv  Detail & Related papers  (2026-03-20T02:12:43Z)
• Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking [51.31378940976401]
  Existing RGB-Event tracking approaches fail to fully exploit the unique advantages of event cameras.<n>We propose a novel tracking framework that performs early fusion in the frequency domain, enabling effective aggregation of high-frequency information from the event modality.<n>Experiments on three widely used RGB-Event tracking benchmark datasets, including FE108, FELT, and COESOT, demonstrate the high performance and efficiency of our method.
  arXiv  Detail & Related papers  (2026-01-03T01:10:17Z)
• Collaborating Vision, Depth, and Thermal Signals for Multi-Modal Tracking: Dataset and Algorithm [103.36490810025752]
  Existing multi-modal object tracking approaches primarily focus on dual-modal paradigms, such as RGB-Depth or RGB-Thermal.<n>This work introduces a novel multi-modal tracking task that leverages three complementary modalities, including visible RGB, Depth (D), and Thermal Infrared (TIR)<n>We propose a novel multi-modal tracker, dubbed RDTTrack, which integrates tri-modal information for robust tracking by leveraging a pretrained RGB-only tracking model.
  arXiv  Detail & Related papers  (2025-09-29T13:05:15Z)
• Leveraging RGB Images for Pre-Training of Event-Based Hand Pose Estimation [64.8814078041756]
  RPEP is the first pre-training method for event-based 3D hand pose estimation using labeled RGB images and unpaired, unlabeled event data.<n>Our model significantly outperforms state-of-the-art methods on real event data, achieving up to 24% improvement on EvRealHands.
  arXiv  Detail & Related papers  (2025-09-21T07:07:49Z)
• Learning Flow-Guided Registration for RGB-Event Semantic Segmentation [22.996619370156584]
  Event cameras capture microsecond-level motion cues that complement RGB sensors.<n>We recast RGB-Event segmentation from fusion to registration.<n>We propose BRENet, a novel flow-guided bidirectional framework that adaptively matches correspondence between the asymmetric modalities.
  arXiv  Detail & Related papers  (2025-05-02T19:19:58Z)
• Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset [65.76480665062363]
  Human Activity Recognition primarily relied on traditional RGB cameras to achieve high-performance activity recognition.<n>Challenges in real-world scenarios, such as insufficient lighting and rapid movements, inevitably degrade the performance of RGB cameras.<n>In this work, we rethink human activity recognition by combining the RGB and event cameras.
  arXiv  Detail & Related papers  (2025-04-08T09:14:24Z)
• RGB-Sonar Tracking Benchmark and Spatial Cross-Attention Transformer Tracker [4.235252053339947]
  This paper introduces a new challenging RGB-Sonar (RGB-S) tracking task. It investigates how to achieve efficient tracking of an underwater target through the interaction of RGB and sonar modalities.
  arXiv  Detail & Related papers  (2024-06-11T12:01:11Z)
• Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera [8.673063170884591]
  EOLO is a novel object detection framework that achieves robust and efficient all-day detection by fusing both RGB and event modalities. Our EOLO framework is built based on a lightweight spiking neural network (SNN) to efficiently leverage the asynchronous property of events.
  arXiv  Detail & Related papers  (2023-09-17T15:14:01Z)
• SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition [40.107228252231515]
  We propose to recognize patterns by fusing RGB frames and event streams simultaneously.<n>Due to the scarce of RGB-Event based classification dataset, we also propose a large-scale PokerEvent dataset.
  arXiv  Detail & Related papers  (2023-08-08T16:15:35Z)
• CoMAE: Single Model Hybrid Pre-training on Small-Scale RGB-D Datasets [50.6643933702394]
  We present a single-model self-supervised hybrid pre-training framework for RGB and depth modalities, termed as CoMAE. Our CoMAE presents a curriculum learning strategy to unify the two popular self-supervised representation learning algorithms: contrastive learning and masked image modeling.
  arXiv  Detail & Related papers  (2023-02-13T07:09:45Z)
• RGB-Event Fusion for Moving Object Detection in Autonomous Driving [3.5397758597664306]
  Moving Object Detection (MOD) is a critical vision task for successfully achieving safe autonomous driving. Recent advances in sensor technologies, especially the Event camera, can naturally complement the conventional camera approach to better model moving objects. We propose RENet, a novel RGB-Event fusion Network, that jointly exploits the two complementary modalities to achieve more robust MOD.
  arXiv  Detail & Related papers  (2022-09-17T12:59:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.