VISTA: Technical Report for the Ego4D Short-Term Object Interaction Anticipation at EgoVis 2026

• URL: http://arxiv.org/abs/2605.20901v1
• Date: Wed, 20 May 2026 08:42:56 GMT
• Title: VISTA: Technical Report for the Ego4D Short-Term Object Interaction Anticipation at EgoVis 2026
• Abstract summary: Given an egocentric video timestamp, the task requires anticipating the next human-object interaction.<n>ViSTA follows a StillFast-style design that combines object-centric spatial detection with short-horizon temporal context.<n>Experiments show that VISTA achieves first place in the EgoVis 2026 Ego4D STA Challenge.
• Score: 70.67609435035631
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose VISTA, a V-JEPA Integrated StillFast Temporal Anticipator for the Ego4D Short-Term Object Interaction Anticipation (STA) Challenge at EgoVis 2026. Given an egocentric video timestamp, the task requires anticipating the next human-object interaction, including the future active object's bounding box, noun category, verb category, time-to-contact, and confidence score. VISTA follows a StillFast-style design that combines object-centric spatial detection with short-horizon temporal context. Specifically, a COCO-pretrained Faster R-CNN ResNet-50 FPN detector generates object proposals from the last observed high-resolution frame, while a frozen V-JEPA 2.1 temporal branch extracts clip-level egocentric context from the observed video. The temporal representation is injected into the detection pathway through feature modulation and ROI-level context fusion. The fused proposal features are then passed to multi-head STA predictors for box refinement, noun classification, verb classification, time-to-contact regression, and interaction confidence estimation. For the final submission, we further ensemble complementary predictions to improve robustness. Experimental results on the official challenge server show that VISTA achieves first place in the EgoVis 2026 Ego4D STA Challenge. Our code will be released at https://github.com/CorrineQiu/VISTA.

Related papers

• EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understanding [12.811023931580523]
  EgoSAT is the first comprehensive benchmark for egocentric video reasoning in streaming settings.<n>It is designed to evaluate the capabilities of modern vision-language models (VLMs)<n>EgoSAT contains 1,997 unique videos spanning 165 hours of egocentric footage and around 4,800 high-quality question-answer pairs.
  arXiv  Detail & Related papers  (2026-06-23T10:59:25Z)
• FROST-STA: Frozen Dense Features for the Ego4D Short-Term Object Interaction Anticipation [4.781417767575192]
  This report describes FROST-STA, our submission to the Ego4D Short-Term Object Interaction Anticipation (STA) Challenge at EgoVis 2026.<n>For each query time, the model produces a ranked set of structured hypotheses containing an active-object box, noun label, verb label, time-to-contact (TTC), and confidence.<n> FROST-STA builds on the V-JEPA 2.1 STA evaluation protocol, but adapts it to the challenge by using object-centric decoding, multi-head prediction, and a submission-oriented training and ensembling recipe.
  arXiv  Detail & Related papers  (2026-05-30T12:07:32Z)
• EgoAction: Egocentric Action Composition with Reliability-Aware Temporal Fusion for the EPIC-KITCHENS Action Detection Challenge at CVPR 2026 [69.56534058291463]
  We formulate our submission as EgoAction, a unified decoupled detection and fusion pipeline.<n>The pipeline uses EPIC-finetuned VideoMAE-L features, trains separate noun and verb temporal detectors with causal temporal modeling.<n>EgoAction provides a compact and reproducible system for egocentric temporal action detection.
  arXiv  Detail & Related papers  (2026-05-23T10:05:56Z)
• Integrating Affordances and Attention models for Short-Term Object Interaction Anticipation [17.55088626523059]
  Short Term object-interaction Anticipation is fundamental for wearable assistants to understand user goals and provide timely assistance.<n>In this work, we present a method to improve the performance of STA predictions.<n>Our results show significant improvements on Overall Top-5 mAP, with gain up to +23p.p on Ego4D and +31p.p on a novel set of curated EPIC-Kitchens STA labels.
  arXiv  Detail & Related papers  (2026-02-16T15:29:04Z)
• Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025 [77.414837862995]
  We present a novel three-stage framework developed for the Ego4D Long-Term Action Anticipation task.<n>Inspired by recent advances in foundation models, our method consists of three stages: feature extraction, action recognition, and long-term action anticipation.<n>Our framework achieves first place in this challenge at CVPR 2025, establishing a new state-of-the-art in long-term action prediction.
  arXiv  Detail & Related papers  (2025-06-03T07:36:52Z)
• Harnessing Temporal Causality for Advanced Temporal Action Detection [53.654457142657236]
  We introduce CausalTAD, which combines causal attention and causal Mamba to achieve state-of-the-art performance on benchmarks. We ranked 1st in the Action Recognition, Action Detection, and Audio-Based Interaction Detection tracks at the EPIC-Kitchens Challenge 2024, and 1st in the Moment Queries track at the Ego4D Challenge 2024.
  arXiv  Detail & Related papers  (2024-07-25T06:03:02Z)
• Short-term Object Interaction Anticipation with Disentangled Object Detection @ Ego4D Short Term Object Interaction Anticipation Challenge [11.429137967096935]
  Short-term object interaction anticipation is an important task in egocentric video analysis. Our proposed method, SOIA-DOD, effectively decomposes it into 1) detecting active object and 2) classifying interaction and predicting their timing. Our method first detects all potential active objects in the last frame of egocentric video by fine-tuning a pre-trained YOLOv9.
  arXiv  Detail & Related papers  (2024-07-08T08:13:16Z)
• ZARRIO @ Ego4D Short Term Object Interaction Anticipation Challenge: Leveraging Affordances and Attention-based models for STA [10.144283429670807]
  Short-Term object-interaction Anticipation (STA) consists of detecting the location of the next-active objects, the noun and verb categories of the interaction, and the time to contact from the observation of egocentric video. We propose STAformer, a novel attention-based architecture integrating frame-guided temporal pooling, dual image-video attention, and multi-scale feature fusion to support STA predictions from an image-input video pair.
  arXiv  Detail & Related papers  (2024-07-05T09:16:30Z)
• AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipation [14.734158936250918]
  Short-Term object-interaction Anticipation is fundamental for wearable assistants or human robot interaction to understand user goals. We improve the performance of STA predictions with two contributions. First, we propose STAformer, a novel attention-based architecture integrating frame guided temporal pooling, dual image-video attention, and multiscale feature fusion. Second, we predict interaction hotspots from the observation of hands and object trajectories, increasing confidence in STA predictions localized around the hotspot.
  arXiv  Detail & Related papers  (2024-06-03T10:57:18Z)
• Recurrent Vision Transformers for Object Detection with Event Cameras [62.27246562304705]
  We present Recurrent Vision Transformers (RVTs), a novel backbone for object detection with event cameras. RVTs can be trained from scratch to reach state-of-the-art performance on event-based object detection. Our study brings new insights into effective design choices that can be fruitful for research beyond event-based vision.
  arXiv  Detail & Related papers  (2022-12-11T20:28:59Z)
• A Prospective Study on Sequence-Driven Temporal Sampling and Ego-Motion Compensation for Action Recognition in the EPIC-Kitchens Dataset [68.8204255655161]
  Action recognition is one of the top-challenging research fields in computer vision. ego-motion recorded sequences have become of important relevance. The proposed method aims to cope with it by estimating this ego-motion or camera motion.
  arXiv  Detail & Related papers  (2020-08-26T14:44:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.