InterFuserDVS: Event-Enhanced Sensor Fusion for Safe RL-Based Decision Making

• URL: http://arxiv.org/abs/2605.04355v1
• Date: Tue, 05 May 2026 23:24:45 GMT
• Title: InterFuserDVS: Event-Enhanced Sensor Fusion for Safe RL-Based Decision Making
• Abstract summary: We introduce a novel token-based fusion strategy that incorporates accumulated event frames into the transformer-based backbone of InterFuser.<n>We evaluate our approach on the Car Learning to Act (CARLA) Leaderboard benchmarks, demonstrating that the inclusion of DVS improves the robustness of the driving agent.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous driving systems rely heavily on robust sensor fusion to perceive complex envi- ronments. Traditional setups using RGB cameras and LiDAR often struggle in high-dynamic- range scenes or high-speed scenarios due to motion blur and latency. Dynamic Vision Sensors (DVS), or event cameras, offer a paradigm shift by capturing asynchronous brightness changes with microsecond temporal resolution and high dynamic range. In this paper, we propose an extended architecture of the state-of-the-art InterFuser model, integrating DVS as an additional modality to enhance perception reliability. We introduce a novel token-based fusion strategy that incorporates accumulated event frames into the transformer-based backbone of InterFuser. Our method leverages the complementary nature of RGB, LiDAR, and DVS data. We evaluate our approach on the Car Learning to Act (CARLA) Leaderboard benchmarks, demonstrating that the inclusion of DVS improves the robustness of the driving agent, achieving a competitive Driving Score of 77.2 and a superior Route Completion of 100%. The results indicate that event-based vision is a promising direction for improving safety and performance in adverse lighting and dynamic conditions.

Related papers

• DeepIPCv3: Event-Aware Multi-Modal Sensor Fusion for Sudden Pedestrian Crossing Avoidance [4.219150964619931]
  DeepIPCv3 is a novel multi-modal autonomous navigation framework.<n>It synergizes the dense 3D spatial geometry of LiDAR point clouds with the microsecond-level asynchronous event streams of a Dynamic Vision Sensor.<n>The proposed LiDAR and DVS fusion yields the lowest trajectory and control command errors, enabling highly reactive, mathematically bounded evasive maneuvers.
  arXiv  Detail & Related papers  (2026-05-31T14:59:01Z)
• EventGait: Towards Robust Gait Recognition with Event Streams [50.890621860023]
  Event cameras offer microsecond temporal resolution and high dynamic range, naturally capturing robust dynamic cues and suppressing static noise.<n>We propose textbfEventGait, an end-to-end dual-stream framework that separately models motion and shape while preserving the advantages of events.<n>Our approach sets a new state of the art on both synthesized and real-world event-based gait benchmarks, highlighting the robustness and potential of event-driven gait analysis.
  arXiv  Detail & Related papers  (2026-05-21T08:12:04Z)
• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• Digital Twin-based Control Co-Design of Full Vehicle Active Suspensions via Deep Reinforcement Learning [8.45003507603975]
  Digital twins (DTs) and deep reinforcement learning (DRL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle.<n>This work presents a DT-based control co-design (CCD) framework for full-vehicle active suspensions.<n>We jointly optimize physical suspension components and control policies under varying driver behaviors and environmental uncertainties.
  arXiv  Detail & Related papers  (2025-12-03T15:41:35Z)
• Ensemble-Based Event Camera Place Recognition Under Varying Illumination [15.33833908429706]
  Event cameras provide a high dynamic range and low latency, offering greater robustness to rapid motion and challenging lighting conditions.<n>We introduce an ensemble-based approach to event camera place recognition that combines sequence-matched results from multiple event-to-frame reconstructions, VPR feature extractors, and temporal resolutions.<n>Unlike previous event-based ensemble methods, which only utilise temporal resolution, our broader fusion strategy delivers significantly improved robustness under varied lighting conditions.
  arXiv  Detail & Related papers  (2025-09-02T05:17:07Z)
• Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance [12.513296074529727]
  This paper proposes the Real-time Edge-based Autonomous Co-pilot Trajectory planner (REACT) for autonomous driving.<n>REACT is a V2X-integrated trajectory optimization framework for AD based on a fine-tuned lightweight Vision-Language Model (VLM)<n> evaluated on the DeepAccident benchmark, REACT achieves state-of-the-art performance, a 77% collision rate reduction, a 48.2% Video Panoptic Quality (VPQ), and a 0.57-second inference latency on the Jetson AGX Orin.
  arXiv  Detail & Related papers  (2025-08-01T20:16:04Z)
• SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving [51.47621083057114]
  SOLVE is an innovative framework that synergizes Vision-Language Models with end-to-end (E2E) models to enhance autonomous vehicle planning.<n>Our approach emphasizes knowledge sharing at the feature level through a shared visual encoder, enabling comprehensive interaction between VLM and E2E components.
  arXiv  Detail & Related papers  (2025-05-22T15:44:30Z)
• Video-based Traffic Light Recognition by Rockchip RV1126 for Autonomous Driving [19.468567166834585]
  Real-time traffic light recognition is fundamental for autonomous driving safety and navigation in urban environments.<n>We present textitViTLR, a novel video-based end-to-end neural network that processes multiple consecutive frames to achieve robust traffic light detection and state classification.<n>We have successfully integrated textitViTLR into an ego-lane traffic light recognition system using HD maps for autonomous driving applications.
  arXiv  Detail & Related papers  (2025-03-31T11:27:48Z)
• Multi-modal Multi-platform Person Re-Identification: Benchmark and Method [58.59888754340054]
  MP-ReID is a novel dataset designed specifically for multi-modality and multi-platform ReID.<n>This benchmark compiles data from 1,930 identities across diverse modalities, including RGB, infrared, and thermal imaging.<n>We introduce Uni-Prompt ReID, a framework with specific-designed prompts, tailored for cross-modality and cross-platform scenarios.
  arXiv  Detail & Related papers  (2025-03-21T12:27:49Z)
• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving.<n>Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner.<n>Experiments conducted on nuScenes and Bench2Drive datasets demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)
• Self-Supervised Scene Dynamic Recovery from Rolling Shutter Images and Events [63.984927609545856]
  Event-based Inter/intra-frame Compensator (E-IC) is proposed to predict the per-pixel dynamic between arbitrary time intervals. We show that the proposed method achieves state-of-the-art and shows remarkable performance for event-based RS2GS inversion in real-world scenarios.
  arXiv  Detail & Related papers  (2023-04-14T05:30:02Z)
• RGB-Event Fusion for Moving Object Detection in Autonomous Driving [3.5397758597664306]
  Moving Object Detection (MOD) is a critical vision task for successfully achieving safe autonomous driving. Recent advances in sensor technologies, especially the Event camera, can naturally complement the conventional camera approach to better model moving objects. We propose RENet, a novel RGB-Event fusion Network, that jointly exploits the two complementary modalities to achieve more robust MOD.
  arXiv  Detail & Related papers  (2022-09-17T12:59:08Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.