Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation

• URL: http://arxiv.org/abs/2609.16811v1
• Date: Tue, 15 Sep 2026 08:17:37 GMT
• Title: Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation
• Abstract summary: We propose a simple, painless, and scalable image-to-event pretraining framework that transfers high-order semantic structures from images to events.<n>Hyper-RED uses hypergraphs to model and align high-order semantic associations among multiple image and event tokens.<n> Experiments on three tasks across five event datasets demonstrate consistent scaling from ViT-S to ViT-L and state-of-the-art performance.
• Score: 68.90987453924909
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event cameras have shown great potential for robust visual perception, yet scaling event representation learning remains challenging due to the scarcity of large-scale annotated event data. Pretrained image models provide scalable semantic supervision, but existing image-to-event methods rely on rigid pixel-wise or token-wise alignment that overlooks modality discrepancies in texture, density, and appearance, potentially causing semantic collapse and limiting transferability. To address this issue, we propose Hyper-RED, a simple, painless, and scalable image-to-event pretraining framework that transfers high-order semantic structures from images to events. Hyper-RED uses hypergraphs to model and align high-order semantic associations among multiple image and event tokens, enabling cross-modal knowledge transfer while accommodating modality-specific differences rather than enforcing rigid one-to-one correspondence. Specifically, given a paired event--image sample, Hyper-RED leverages DINOv3 to extract spatial token representations and constructs image, event, and cross-modal semantic hypergraphs, where each hyperedge connects multiple semantically correlated tokens. We further introduce a hypergraph relational distillation loss that imposes complementary intra- and cross-modal constraints, enabling the event encoder to inherit image-derived semantic organization while preserving local relational consistency and event-specific characteristics. Experiments on three tasks across five event datasets demonstrate consistent scaling from ViT-S to ViT-L and state-of-the-art performance (Fig.1). The code is available at: https://github.com/meisenwang/Hyper--RED.

Related papers

• Generative Event Pretraining with Foundation Model Alignment [25.86429472344715]
  Event cameras provide robust visual signals under fast motion and challenging illumination conditions.<n>Their unique sensing characteristics and limited labeled data make it challenging to train event-based visual foundation models.<n>We propose GEP (Generative Event Pretraining), a two-stage framework that transfers semantic knowledge learned from internet-scale image datasets to event data.
  arXiv  Detail & Related papers  (2026-03-24T10:10:34Z)
• Scaling Dense Event-Stream Pretraining from Visual Foundation Models [112.44243079477137]
  We launch a novel self-supervised pretraining method that distills visual foundation models (VFMs) to push the boundaries of event representation at scale.<n>We curate an extensive synchronized image-event collection to amplify cross-modal alignment.<n>We extend the alignment objective to semantic structures provided off-the-shelf by VFMs, indicating a broader receptive field and stronger supervision.
  arXiv  Detail & Related papers  (2026-03-04T12:06:09Z)
• SGDiff: Scene Graph Guided Diffusion Model for Image Collaborative SegCaptioning [53.638998508418545]
  This paper introduces a new task Image Collaborative and Captioning'' (SegCaptioning)<n>SegCaptioning aims to translate a straightforward prompt, like a bounding box around an object, into diverse semantic interpretations represented by (caption, masks) pairs.<n>This task poses significant challenges, including accurately capturing a user's intention from a minimal prompt while simultaneously predicting multiple semantically aligned caption words and masks.
  arXiv  Detail & Related papers  (2025-12-01T18:33:04Z)
• EvRainDrop: HyperGraph-guided Completion for Effective Frame and Event Stream Aggregation [13.258464981482788]
  Event cameras produce asynchronous event streams that are spatially sparse yet temporally dense.<n>We propose a novel hypergraph-guided event stream completion mechanism, which connects tokens across different times and locations.<n>The proposed method can flexibly incorporate RGB tokens as nodes in the hypergraph within this completion framework, enabling multi-modal hypergraph-based information completion.
  arXiv  Detail & Related papers  (2025-11-26T14:30:04Z)
• SoftHGNN: Soft Hypergraph Neural Networks for General Visual Recognition [19.88933136559363]
  Hypergraphs extend conventional graphs by modeling high-order interactions.<n>Existing hypergraph neural networks typically rely on static and hard hyperedge assignments.<n>We present Soft Hypergraph Neural Networks (SoftHGNNs), which extend the methodology of hypergraph computation.
  arXiv  Detail & Related papers  (2025-05-21T10:01:30Z)
• Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks [53.67497327319569]
  We introduce a novel neural rendering technique to solve image-to-3D from a single view. Our approach employs the signed distance function as the surface representation and incorporates generalizable priors through geometry-encoding volumes and HyperNetworks. Our experiments show the advantages of our proposed approach with consistent results and rapid generation.
  arXiv  Detail & Related papers  (2023-12-24T08:42:37Z)
• EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding [7.797154022794006]
  EventBind is a novel framework that unleashes the potential of vision-language models (VLMs) for event-based recognition. We first introduce a novel event encoder that subtly models the temporal information from events. We then design a text encoder that generates content prompts and utilizes hybrid text prompts to enhance EventBind's generalization ability.
  arXiv  Detail & Related papers  (2023-08-06T15:05:42Z)
• Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence [88.00004819064672]
  Diffusion Hyperfeatures is a framework for consolidating multi-scale and multi-timestep feature maps into per-pixel feature descriptors. Our method achieves superior performance on the SPair-71k real image benchmark.
  arXiv  Detail & Related papers  (2023-05-23T17:58:05Z)
• Event Camera Data Pre-training [14.77724035068357]
  Our model is a self-supervised learning framework, and uses paired event camera data and natural RGB images for training. We achieve top-1 accuracy at 64.83% on the N-ImageNet dataset.
  arXiv  Detail & Related papers  (2023-01-05T06:32:50Z)
• Intriguing Properties of Vision Transformers [114.28522466830374]
  Vision transformers (ViT) have demonstrated impressive performance across various machine vision problems. We systematically study this question via an extensive set of experiments and comparisons with a high-performing convolutional neural network (CNN) We show effective features of ViTs are due to flexible receptive and dynamic fields possible via the self-attention mechanism.
  arXiv  Detail & Related papers  (2021-05-21T17:59:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.