EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation

• URL: http://arxiv.org/abs/2603.09385v1
• Date: Tue, 10 Mar 2026 08:57:51 GMT
• Title: EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation
• Abstract summary: EventVGGT is a novel framework that explicitly models the event stream as a coherent video sequence.<n>It consistently outperforms existing methods, reducing the absolute mean depth at 30m by over 53%.<n>It also exhibits robust zero-shot error on the unseen DENSE and MVSEC datasets.
• Score: 24.595306617729012
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Event cameras offer superior sensitivity to high-speed motion and extreme lighting, making event-based monocular depth estimation a promising approach for robust 3D perception in challenging conditions. However, progress is severely hindered by the scarcity of dense depth annotations. While recent annotation-free approaches mitigate this by distilling knowledge from Vision Foundation Models (VFMs), a critical limitation persists: they process event streams as independent frames. By neglecting the inherent temporal continuity of event data, these methods fail to leverage the rich temporal priors encoded in VFMs, ultimately yielding temporally inconsistent and less accurate depth predictions. To address this, we introduce EventVGGT, a novel framework that explicitly models the event stream as a coherent video sequence. To the best of our knowledge, we are the first to distill spatio-temporal and multi-view geometric priors from the Visual Geometry Grounded Transformer (VGGT) into the event domain. We achieve this via a comprehensive tri-level distillation strategy: (i) Cross-Modal Feature Mixture (CMFM) bridges the modality gap at the output level by fusing RGB and event features to generate auxiliary depth predictions; (ii) Spatio-Temporal Feature Distillation (STFD) distills VGGT's powerful spatio-temporal representations at the feature level; and (iii) Temporal Consistency Distillation (TCD) enforces cross-frame coherence at the temporal level by aligning inter-frame depth changes. Extensive experiments demonstrate that EventVGGT consistently outperforms existing methods -- reducing the absolute mean depth error at 30m by over 53\% on EventScape (from 2.30 to 1.06) -- while exhibiting robust zero-shot generalization on the unseen DENSE and MVSEC datasets.

Related papers

• FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection [0.0]
  Event cameras offer inherent advantages in high-speed and high-dynamic-range scenarios.<n>Current approaches partition the accumulation window into fixed temporal sub-bins.<n>We propose FATE, a unified framework built upon a novel spatial framework.
  arXiv  Detail & Related papers  (2026-06-15T22:32:09Z)
• Rethinking Event-Based Object Dtection through Representation-Level Temporal Aggregation and Model-Level Hypergraph Reasoning [65.08890312027314]
  Event cameras provide microsecond-level temporal resolution, low latency, and high dynamic range.<n>Event Dual Temporal-Relational Aggregation Detector (Ev-DTAD) integrates representation-level temporal encoding with model-level temporal-hypergraph reasoning.<n>We show that Ev-DTAD achieves a competitive accuracy-efficiency trade-off, validating the complementarity between compact temporal representation and temporal-hypergraph feature reasoning.
  arXiv  Detail & Related papers  (2026-05-09T09:20:34Z)
• RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow [51.43025173196566]
  Monocular scene flow estimation aims to recover dense 3D motion from image sequences.<n>We propose RAFT-MSF++, a self-supervised multi-frame framework that fuses temporal features to jointly estimate depth and scene flow.<n>Experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions.
  arXiv  Detail & Related papers  (2026-04-21T11:32:49Z)
• Towards Video Anomaly Detection from Event Streams: A Baseline and Benchmark Datasets [71.53287557600177]
  We take the first major step toward establishing event-based anomaly detection as a unified research direction.<n>We first construct multiple eventstream based benchmarks for video anomaly detection, featuring synchronized event and RGB recordings.<n>We then propose an EVent-centric Video Anomaly Detection framework, namely EWAD, with three key innovations.
  arXiv  Detail & Related papers  (2026-03-26T03:33:33Z)
• Scaling Dense Event-Stream Pretraining from Visual Foundation Models [112.44243079477137]
  We launch a novel self-supervised pretraining method that distills visual foundation models (VFMs) to push the boundaries of event representation at scale.<n>We curate an extensive synchronized image-event collection to amplify cross-modal alignment.<n>We extend the alignment objective to semantic structures provided off-the-shelf by VFMs, indicating a broader receptive field and stronger supervision.
  arXiv  Detail & Related papers  (2026-03-04T12:06:09Z)
• Temporal-Guided Visual Foundation Models for Event-Based Vision [40.997738547677066]
  Event cameras offer unique advantages for vision tasks in challenging environments.<n>The potential of leveraging modern Visual Foundation Models pretrained on image data remains under-explored for event-based vision.<n>We propose TemporalGuided-FM (TGVFM), a novel framework that integrates Visual Foundation Models with our temporal context fusion.
  arXiv  Detail & Related papers  (2025-11-09T05:45:25Z)
• Depth AnyEvent: A Cross-Modal Distillation Paradigm for Event-Based Monocular Depth Estimation [47.90167568304715]
  Event cameras capture sparse, high-temporal-resolution visual information.<n>The lack of large datasets with dense ground-truth depth annotations hinders learning-based monocular depth estimation from event data.<n>We propose a cross-modal distillation paradigm to generate dense proxy labels leveraging a Vision Foundation Model (VFM)<n>Our strategy requires an event stream spatially aligned with RGB frames, a simple setup even available off-the-shelf, and exploits the robustness of large-scale VFMs.
  arXiv  Detail & Related papers  (2025-09-18T17:59:51Z)
• HyPCV-Former: Hyperbolic Spatio-Temporal Transformer for 3D Point Cloud Video Anomaly Detection [1.475698751142657]
  HyV-Former achieves state-of-the-art anomaly detection across multiple anomaly categories, with a 7% improvement on the TIMo dataset and a 5.6% gain on the DAD dataset.
  arXiv  Detail & Related papers  (2025-08-01T09:50:20Z)
• SuperFlow++: Enhanced Spatiotemporal Consistency for Cross-Modal Data Pretraining [62.433137130087445]
  SuperFlow++ is a novel framework that integrates pretraining and downstream tasks using consecutive camera pairs.<n>We show that SuperFlow++ outperforms state-of-the-art methods across diverse tasks and driving conditions.<n>With strong generalizability and computational efficiency, SuperFlow++ establishes a new benchmark for data-efficient LiDAR-based perception in autonomous driving.
  arXiv  Detail & Related papers  (2025-03-25T17:59:57Z)
• Temporal Residual Guided Diffusion Framework for Event-Driven Video Reconstruction [23.493813870675197]
  Event-based video reconstruction has garnered increasing attention due to its advantages, such as high dynamic range and rapid motion capture capabilities. Current methods often prioritize the extraction of temporal information from continuous event flow, leading to an overemphasis on low-frequency texture features in the scene. We introduce a novel approach, the Temporal Residual Guided Diffusion Framework, which effectively leverages both temporal and frequency-based event priors.
  arXiv  Detail & Related papers  (2024-07-15T11:48:57Z)
• On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation [56.97699793236174]
  We study two kinds of robust cross-view consistency in this paper. We exploit the temporal coherence in both depth feature space and 3D voxel space for self-supervised monocular depth estimation. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques.
  arXiv  Detail & Related papers  (2022-09-19T03:46:13Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.