Untrimmed Action Anticipation

• URL: http://arxiv.org/abs/2202.04132v1
• Date: Tue, 8 Feb 2022 20:20:08 GMT
• Title: Untrimmed Action Anticipation
• Authors: Ivan Rodin, Antonino Furnari, Dimitrios Mavroeidis and Giovanni Maria Farinella
• Abstract summary: Egocentric action anticipation consists in predicting a future action the camera wearer will perform from egocentric video. Current approaches assume that the input videos are "trimmed", meaning that a short video sequence is sampled a fixed time before the beginning of the action. We argue that, despite the recent advances in the field, trimmed action anticipation has a limited applicability in real-world scenarios.
• Score: 20.630139085937586
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Egocentric action anticipation consists in predicting a future action the camera wearer will perform from egocentric video. While the task has recently attracted the attention of the research community, current approaches assume that the input videos are "trimmed", meaning that a short video sequence is sampled a fixed time before the beginning of the action. We argue that, despite the recent advances in the field, trimmed action anticipation has a limited applicability in real-world scenarios where it is important to deal with "untrimmed" video inputs and it cannot be assumed that the exact moment in which the action will begin is known at test time. To overcome such limitations, we propose an untrimmed action anticipation task, which, similarly to temporal action detection, assumes that the input video is untrimmed at test time, while still requiring predictions to be made before the actions actually take place. We design an evaluation procedure for methods designed to address this novel task, and compare several baselines on the EPIC-KITCHENS-100 dataset. Experiments show that the performance of current models designed for trimmed action anticipation is very limited and more research on this task is required.

Related papers

• Action Anticipation from SoccerNet Football Video Broadcasts [84.87912817065506]
  We introduce the task of action anticipation for football broadcast videos. We predict future actions in unobserved future frames within a five- or ten-second anticipation window. Our work will enable applications in automated broadcasting, tactical analysis, and player decision-making.
  arXiv  Detail & Related papers  (2025-04-16T12:24:33Z)
• About Time: Advances, Challenges, and Outlooks of Action Understanding [57.76390141287026]
  This survey comprehensively reviews advances in uni- and multi-modal action understanding across a range of tasks. We focus on prevalent challenges, overview widely adopted datasets, and survey seminal works with an emphasis on recent advances.
  arXiv  Detail & Related papers  (2024-11-22T18:09:27Z)
• From Recognition to Prediction: Leveraging Sequence Reasoning for Action Anticipation [30.161471749050833]
  We propose a novel end-to-end video modeling architecture that utilizes attention mechanisms, named Anticipation via Recognition and Reasoning (ARR) ARR decomposes the action anticipation task into action recognition and reasoning tasks, and effectively learns the statistical relationship between actions by next action prediction (NAP) In addition, to address the challenge of relationship modeling that requires extensive training data, we propose an innovative approach for the unsupervised pre-training of the decoder.
  arXiv  Detail & Related papers  (2024-08-05T18:38:29Z)
• Harnessing Temporal Causality for Advanced Temporal Action Detection [53.654457142657236]
  We introduce CausalTAD, which combines causal attention and causal Mamba to achieve state-of-the-art performance on benchmarks. We ranked 1st in the Action Recognition, Action Detection, and Audio-Based Interaction Detection tracks at the EPIC-Kitchens Challenge 2024, and 1st in the Moment Queries track at the Ego4D Challenge 2024.
  arXiv  Detail & Related papers  (2024-07-25T06:03:02Z)
• Inductive Attention for Video Action Anticipation [16.240254363118016]
  We propose an inductive attention model, dubbed IAM, which leverages the current prior predictions as the query to infer future action. Our method consistently outperforms the state-of-the-art anticipation models on multiple large-scale egocentric video datasets.
  arXiv  Detail & Related papers  (2022-12-17T09:51:17Z)
• The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction [104.628661890361]
  Early action prediction deals with inferring the ongoing action from partially-observed videos, typically at the outset of the video. We propose a bottleneck-based attention model that captures the evolution of the action, through progressive sampling over fine-to-coarse scales.
  arXiv  Detail & Related papers  (2022-04-28T08:21:09Z)
• Video Action Detection: Analysing Limitations and Challenges [70.01260415234127]
  We analyze existing datasets on video action detection and discuss their limitations. We perform a biasness study which analyzes a key property differentiating videos from static images: the temporal aspect. Such extreme experiments show existence of biases which have managed to creep into existing methods inspite of careful modeling.
  arXiv  Detail & Related papers  (2022-04-17T00:42:14Z)
• Towards Streaming Egocentric Action Anticipation [23.9991007631236]
  Egocentric action anticipation is the task of predicting the future actions a camera wearer will likely perform based on past video observations. Current evaluation schemes assume that predictions can be made offline, and hence that computational resources are not limited. We propose a streaming'' egocentric action anticipation evaluation protocol which explicitly considers model runtime for performance assessment.
  arXiv  Detail & Related papers  (2021-10-11T16:22:56Z)
• Review of Video Predictive Understanding: Early ActionRecognition and Future Action Prediction [39.966828592322315]
  Action prediction is a major sub-area of video predictive understanding. Various mathematical tools are widely adopted jointly with computer vision techniques for these two tasks. Structures that rely on deep convolutional neural networks and recurrent neural networks have been extensively proposed for improving the performance of existing vision tasks.
  arXiv  Detail & Related papers  (2021-07-11T22:46:52Z)
• Learning to Anticipate Egocentric Actions by Imagination [60.21323541219304]
  We study the egocentric action anticipation task, which predicts future action seconds before it is performed for egocentric videos. Our method significantly outperforms previous methods on both the seen test set and the unseen test set of the EPIC Kitchens Action Anticipation Challenge.
  arXiv  Detail & Related papers  (2021-01-13T08:04:10Z)
• Revisiting Few-shot Activity Detection with Class Similarity Control [107.79338380065286]
  We present a framework for few-shot temporal activity detection based on proposal regression. Our model is end-to-end trainable, takes into account the frame rate differences between few-shot activities and untrimmed test videos, and can benefit from additional few-shot examples.
  arXiv  Detail & Related papers  (2020-03-31T22:02:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.