Related papers: Long-Term Anticipation of Activities with Cycle Consistency

Long-Term Anticipation of Activities with Cycle Consistency

URL: http://arxiv.org/abs/2009.01142v1
Date: Wed, 2 Sep 2020 15:41:32 GMT
Title: Long-Term Anticipation of Activities with Cycle Consistency
Authors: Yazan Abu Farha, Qiuhong Ke, Bernt Schiele, Juergen Gall
Abstract summary: We propose a framework for anticipating future activities directly from the features of the observed frames and train it in an end-to-end fashion. Our framework achieves state-the-art results on two datasets: the Breakfast dataset and 50Salads.
Score: 90.79357258104417
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: With the success of deep learning methods in analyzing activities in videos, more attention has recently been focused towards anticipating future activities. However, most of the work on anticipation either analyzes a partially observed activity or predicts the next action class. Recently, new approaches have been proposed to extend the prediction horizon up to several minutes in the future and that anticipate a sequence of future activities including their durations. While these works decouple the semantic interpretation of the observed sequence from the anticipation task, we propose a framework for anticipating future activities directly from the features of the observed frames and train it in an end-to-end fashion. Furthermore, we introduce a cycle consistency loss over time by predicting the past activities given the predicted future. Our framework achieves state-of-the-art results on two datasets: the Breakfast dataset and 50Salads.

Related papers

Human Action Anticipation: A Survey [86.415721659234]
The literature on behavior prediction spans various tasks, including action anticipation, activity forecasting, intent prediction, goal prediction, and so on. Our survey aims to tie together this fragmented literature, covering recent technical innovations as well as the development of new large-scale datasets for model training and evaluation.
arXiv Detail & Related papers (2024-10-17T21:37:40Z)
Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
arXiv Detail & Related papers (2022-10-24T07:54:13Z)
Rethinking Learning Approaches for Long-Term Action Anticipation [32.67768331823358]
Action anticipation involves predicting future actions having observed the initial portion of a video. We introduce ANTICIPATR which performs long-term action anticipation. We propose a two-stage learning approach to train a novel transformer-based model.
arXiv Detail & Related papers (2022-10-20T20:07:30Z)
Future Transformer for Long-term Action Anticipation [33.771374384674836]
We propose an end-to-end attention model for action anticipation, dubbed Future Transformer (FUTR) Unlike the previous autoregressive models, the proposed method learns to predict the whole sequence of future actions in parallel decoding. We evaluate our method on two standard benchmarks for long-term action anticipation, Breakfast and 50 Salads, achieving state-of-the-art results.
arXiv Detail & Related papers (2022-05-27T14:47:43Z)
A-ACT: Action Anticipation through Cycle Transformations [89.83027919085289]
We take a step back to analyze how the human capability to anticipate the future can be transferred to machine learning algorithms. A recent study on human psychology explains that, in anticipating an occurrence, the human brain counts on both systems. In this work, we study the impact of each system for the task of action anticipation and introduce a paradigm to integrate them in a learning framework.
arXiv Detail & Related papers (2022-04-02T21:50:45Z)
Review of Video Predictive Understanding: Early ActionRecognition and Future Action Prediction [39.966828592322315]
Action prediction is a major sub-area of video predictive understanding. Various mathematical tools are widely adopted jointly with computer vision techniques for these two tasks. Structures that rely on deep convolutional neural networks and recurrent neural networks have been extensively proposed for improving the performance of existing vision tasks.
arXiv Detail & Related papers (2021-07-11T22:46:52Z)
Panoptic Segmentation Forecasting [71.75275164959953]
Our goal is to forecast the near future given a set of recent observations. We think this ability to forecast, i.e., to anticipate, is integral for the success of autonomous agents. We develop a two-component model: one component learns the dynamics of the background stuff by anticipating odometry, the other one anticipates the dynamics of detected things.
arXiv Detail & Related papers (2021-04-08T17:59:16Z)
Learning to Anticipate Egocentric Actions by Imagination [60.21323541219304]
We study the egocentric action anticipation task, which predicts future action seconds before it is performed for egocentric videos. Our method significantly outperforms previous methods on both the seen test set and the unseen test set of the EPIC Kitchens Action Anticipation Challenge.
arXiv Detail & Related papers (2021-01-13T08:04:10Z)
Group Activity Prediction with Sequential Relational Anticipation Model [28.225918711577314]
We propose a novel approach to predict group activities given the beginning frames with incomplete activity executions. For group activity prediction, the relation evolution of people's activity and their positions over time is an important cue for predicting group activity. Our model explicitly anticipates both activity features and positions by two graph auto-encoders, aiming to learn a discriminative group representation for group activity prediction.
arXiv Detail & Related papers (2020-08-06T03:17:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.