Energy-Aware Imitation Learning for Steering Prediction Using Events and Frames

• URL: http://arxiv.org/abs/2603.28008v1
• Date: Mon, 30 Mar 2026 03:58:47 GMT
• Title: Energy-Aware Imitation Learning for Steering Prediction Using Events and Frames
• Abstract summary: In autonomous driving, relying solely on frame-based cameras can lead to inaccuracies.<n>We introduce a bio-inspired vision sensor known as the event camera.<n>We propose an energy-aware imitation learning framework for steering prediction.
• Score: 39.06716892671148
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In autonomous driving, relying solely on frame-based cameras can lead to inaccuracies caused by factors like long exposure times, high-speed motion, and challenging lighting conditions. To address these issues, we introduce a bio-inspired vision sensor known as the event camera. Unlike conventional cameras, event cameras capture sparse, asynchronous events that provide a complementary modality to mitigate these challenges. In this work, we propose an energy-aware imitation learning framework for steering prediction that leverages both events and frames. Specifically, we design an Energy-driven Cross-modality Fusion Module (ECFM) and an energy-aware decoder to produce reliable and safe predictions. Extensive experiments on two public real-world datasets, DDD20 and DRFuser, demonstrate that our method outperforms existing state-of-the-art (SOTA) approaches. The codes and trained models will be released upon acceptance.

Related papers

• EventDrive: Event Cameras for Vision-Language Driving Intelligence [26.116057266600794]
  Event cameras sense the world through asynchronous brightness changes with microsecond latency and high dynamic range.<n>We present EventDrive, a large-scale benchmark and model suite that unifies event streams, RGB frames, and language supervision across four core dimensions.
  arXiv  Detail & Related papers  (2026-06-16T17:58:40Z)
• EventGait: Towards Robust Gait Recognition with Event Streams [50.890621860023]
  Event cameras offer microsecond temporal resolution and high dynamic range, naturally capturing robust dynamic cues and suppressing static noise.<n>We propose textbfEventGait, an end-to-end dual-stream framework that separately models motion and shape while preserving the advantages of events.<n>Our approach sets a new state of the art on both synthesized and real-world event-based gait benchmarks, highlighting the robustness and potential of event-driven gait analysis.
  arXiv  Detail & Related papers  (2026-05-21T08:12:04Z)
• InterFuserDVS: Event-Enhanced Sensor Fusion for Safe RL-Based Decision Making [0.0]
  We introduce a novel token-based fusion strategy that incorporates accumulated event frames into the transformer-based backbone of InterFuser.<n>We evaluate our approach on the Car Learning to Act (CARLA) Leaderboard benchmarks, demonstrating that the inclusion of DVS improves the robustness of the driving agent.
  arXiv  Detail & Related papers  (2026-05-05T23:24:45Z)
• Learning to Remove Lens Flare in Event Camera [56.9171469873838]
  We present E-DeflareDeflare, the first framework for removing lens flare from event camera data.<n>We first establish the theoretical foundation by deriving a physics-grounded forward model of the non-linear suppression mechanism.<n> Empowered by this benchmark, we design E-DeflareNet, which achieves state-of-the-art restoration performance.
  arXiv  Detail & Related papers  (2025-12-09T18:59:57Z)
• DriveGazen: Event-Based Driving Status Recognition using Conventional Camera [0.4662017507844857]
  We introduce a wearable driving status recognition device and our open-source dataset, along with a new real-time method robust to changes in lighting conditions for identifying driving status from eye observations of drivers.<n>The core of our method is generating event frames from conventional intensity frames, and the other is a newly designed Attention Driving State Network (ADSN)<n>To the best of our knowledge, our method is the first to utilize guide attention spiking neural networks and eye-based event frames generated from conventional cameras for driving status recognition.
  arXiv  Detail & Related papers  (2024-12-16T13:12:11Z)
• EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset [55.12137324648253]
  Event cameras are emerging imaging technology that offers advantages over conventional frame-based imaging sensors in dynamic range and sensing speed. This paper focuses on five event-aided image and video enhancement tasks.
  arXiv  Detail & Related papers  (2023-12-13T15:42:04Z)
• Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks [55.81577205593956]
  Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously. Deep learning (DL) has been brought to this emerging field and inspired active research endeavors in mining its potential.
  arXiv  Detail & Related papers  (2023-02-17T14:19:28Z)
• Event-based Visual Tracking in Dynamic Environments [0.0]
  We propose a framework to take advantage of both event cameras and off-the-shelf deep learning for object tracking. We show that reconstructing event data into intensity frames improves the tracking performance in conditions under which conventional cameras fail to provide acceptable results.
  arXiv  Detail & Related papers  (2022-12-15T12:18:13Z)
• Lasers to Events: Automatic Extrinsic Calibration of Lidars and Event Cameras [67.84498757689776]
  This paper presents the first direct calibration method between event cameras and lidars. It removes dependencies on frame-based camera intermediaries and/or highly-accurate hand measurements.
  arXiv  Detail & Related papers  (2022-07-03T11:05:45Z)
• Monitoring and Adapting the Physical State of a Camera for Autonomous Vehicles [10.490646039938252]
  We propose a generic and task-oriented self-health-maintenance framework for cameras based on data- and physically-grounded models. We implement the framework on a real-world ground vehicle and demonstrate how a camera can adjust its parameters to counter a poor condition. Our framework not only provides a practical ready-to-use solution to monitor and maintain the health of cameras, but can also serve as a basis for extensions to tackle more sophisticated problems.
  arXiv  Detail & Related papers  (2021-12-10T11:14:44Z)
• E$^2$(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition [21.199869051111367]
  Event cameras capture pixel-level intensity changes in the form of "events" N-EPIC-Kitchens is the first event-based camera extension of the large-scale EPIC-Kitchens dataset. We show that event data provides a comparable performance to RGB and optical flow, yet without any additional flow computation at deploy time.
  arXiv  Detail & Related papers  (2021-12-07T09:43:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.