Generative Event Pretraining with Foundation Model Alignment

• URL: http://arxiv.org/abs/2603.23032v1
• Date: Tue, 24 Mar 2026 10:10:34 GMT
• Title: Generative Event Pretraining with Foundation Model Alignment
• Abstract summary: Event cameras provide robust visual signals under fast motion and challenging illumination conditions.<n>Their unique sensing characteristics and limited labeled data make it challenging to train event-based visual foundation models.<n>We propose GEP (Generative Event Pretraining), a two-stage framework that transfers semantic knowledge learned from internet-scale image datasets to event data.
• Score: 25.86429472344715
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event cameras provide robust visual signals under fast motion and challenging illumination conditions thanks to their microsecond latency and high dynamic range. However, their unique sensing characteristics and limited labeled data make it challenging to train event-based visual foundation models (VFMs), which are crucial for learning visual features transferable across tasks. To tackle this problem, we propose GEP (Generative Event Pretraining), a two-stage framework that transfers semantic knowledge learned from internet-scale image datasets to event data while learning event-specific temporal dynamics. First, an event encoder is aligned to a frozen VFM through a joint regression-contrastive objective, grounding event features in image semantics. Second, a transformer backbone is autoregressively pretrained on mixed event-image sequences to capture the temporal structure unique to events. Our approach outperforms state-of-the-art event pretraining methods on a diverse range of downstream tasks, including object recognition, segmentation, and depth estimation. Together, VFM-guided alignment and generative sequence modeling yield a semantically rich, temporally aware event model that generalizes robustly across domains.

Related papers

• EventGait: Towards Robust Gait Recognition with Event Streams [50.890621860023]
  Event cameras offer microsecond temporal resolution and high dynamic range, naturally capturing robust dynamic cues and suppressing static noise.<n>We propose textbfEventGait, an end-to-end dual-stream framework that separately models motion and shape while preserving the advantages of events.<n>Our approach sets a new state of the art on both synthesized and real-world event-based gait benchmarks, highlighting the robustness and potential of event-driven gait analysis.
  arXiv  Detail & Related papers  (2026-05-21T08:12:04Z)
• Scaling Dense Event-Stream Pretraining from Visual Foundation Models [112.44243079477137]
  We launch a novel self-supervised pretraining method that distills visual foundation models (VFMs) to push the boundaries of event representation at scale.<n>We curate an extensive synchronized image-event collection to amplify cross-modal alignment.<n>We extend the alignment objective to semantic structures provided off-the-shelf by VFMs, indicating a broader receptive field and stronger supervision.
  arXiv  Detail & Related papers  (2026-03-04T12:06:09Z)
• Temporal-Guided Visual Foundation Models for Event-Based Vision [40.997738547677066]
  Event cameras offer unique advantages for vision tasks in challenging environments.<n>The potential of leveraging modern Visual Foundation Models pretrained on image data remains under-explored for event-based vision.<n>We propose TemporalGuided-FM (TGVFM), a novel framework that integrates Visual Foundation Models with our temporal context fusion.
  arXiv  Detail & Related papers  (2025-11-09T05:45:25Z)
• Revealing Latent Information: A Physics-inspired Self-supervised Pre-training Framework for Noisy and Sparse Events [25.348660233701708]
  Event camera records data with high temporal resolution and wide dynamic range.<n>Event data is inherently sparse and noisy, mainly reflecting brightness changes.<n>We propose a self-supervised pre-training framework to fully reveal latent information in event data.
  arXiv  Detail & Related papers  (2025-08-07T15:38:36Z)
• GET: Group Event Transformer for Event-Based Vision [82.312736707534]
  Event cameras are a type of novel neuromorphic sen-sor that has been gaining increasing attention. We propose a novel Group-based vision Transformer backbone for Event-based vision, called Group Event Transformer (GET) GET de-couples temporal-polarity information from spatial infor-mation throughout the feature extraction process.
  arXiv  Detail & Related papers  (2023-10-04T08:02:33Z)
• EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields [80.94515892378053]
  EvDNeRF is a pipeline for generating event data and training an event-based dynamic NeRF. NeRFs offer geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions.
  arXiv  Detail & Related papers  (2023-10-03T21:08:41Z)
• SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features [52.213656737672935]
  SpikeMOT is an event-based multi-object tracker. SpikeMOT uses spiking neural networks to extract sparsetemporal features from event streams associated with objects.
  arXiv  Detail & Related papers  (2023-09-29T05:13:43Z)
• Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams [19.957857885844838]
  Event cameras are neuromorphic vision sensors that record a scene as sparse and asynchronous event streams. We propose an attentionaware model named Event Voxel Set Transformer (EVSTr) for efficient representation learning on event streams. Experiments show that EVSTr achieves state-of-the-art performance while maintaining low model complexity.
  arXiv  Detail & Related papers  (2023-03-07T12:48:02Z)
• Masked Event Modeling: Self-Supervised Pretraining for Event Cameras [41.263606382601886]
  Masked Event Modeling (MEM) is a self-supervised framework for events. MEM pretrains a neural network on unlabeled events, which can originate from any event camera recording. Our method reaches state-of-the-art classification accuracy across three datasets.
  arXiv  Detail & Related papers  (2022-12-20T15:49:56Z)
• Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation [57.22705137545853]
  We propose a task transfer method that allows models to be trained directly with labeled images and unlabeled event data. We leverage the generative event model to split event features into content and motion features. Our approach unlocks the vast amount of existing image datasets for the training of event-based neural networks.
  arXiv  Detail & Related papers  (2021-09-06T17:31:37Z)
• Event-LSTM: An Unsupervised and Asynchronous Learning-based Representation for Event-based Data [8.931153235278831]
  Event cameras are activity-driven bio-inspired vision sensors. We propose Event-LSTM, an unsupervised Auto-Encoder architecture made up of LSTM layers. We also push state-of-the-art event de-noising forward by introducing memory into the de-noising process.
  arXiv  Detail & Related papers  (2021-05-10T09:18:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.