Group Activity Prediction with Sequential Relational Anticipation Model
- URL: http://arxiv.org/abs/2008.02441v1
- Date: Thu, 6 Aug 2020 03:17:14 GMT
- Title: Group Activity Prediction with Sequential Relational Anticipation Model
- Authors: Junwen Chen, Wentao Bao, Yu Kong
- Abstract summary: We propose a novel approach to predict group activities given the beginning frames with incomplete activity executions.
For group activity prediction, the relation evolution of people's activity and their positions over time is an important cue for predicting group activity.
Our model explicitly anticipates both activity features and positions by two graph auto-encoders, aiming to learn a discriminative group representation for group activity prediction.
- Score: 28.225918711577314
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we propose a novel approach to predict group activities given
the beginning frames with incomplete activity executions. Existing action
prediction approaches learn to enhance the representation power of the partial
observation. However, for group activity prediction, the relation evolution of
people's activity and their positions over time is an important cue for
predicting group activity. To this end, we propose a sequential relational
anticipation model (SRAM) that summarizes the relational dynamics in the
partial observation and progressively anticipates the group representations
with rich discriminative information. Our model explicitly anticipates both
activity features and positions by two graph auto-encoders, aiming to learn a
discriminative group representation for group activity prediction. Experimental
results on two popularly used datasets demonstrate that our approach
significantly outperforms the state-of-the-art activity prediction methods.
Related papers
- Activity Grammars for Temporal Action Segmentation [71.03141719666972]
temporal action segmentation aims at translating an untrimmed activity video into a sequence of action segments.
This paper introduces an effective activity grammar to guide neural predictions for temporal action segmentation.
Experimental results demonstrate that our method significantly improves temporal action segmentation in terms of both performance and interpretability.
arXiv Detail & Related papers (2023-12-07T12:45:33Z) - Towards Out-of-Distribution Sequential Event Prediction: A Causal
Treatment [72.50906475214457]
The goal of sequential event prediction is to estimate the next event based on a sequence of historical events.
In practice, the next-event prediction models are trained with sequential data collected at one time.
We propose a framework with hierarchical branching structures for learning context-specific representations.
arXiv Detail & Related papers (2022-10-24T07:54:13Z) - ProspectNet: Weighted Conditional Attention for Future Interaction
Modeling in Behavior Prediction [5.520507323174275]
We formulate the end-to-end joint prediction problem as a sequential learning process of marginal learning and joint learning of vehicle behaviors.
We propose ProspectNet, a joint learning block that adopts the weighted attention score to model the mutual influence between interactive agent pairs.
We show that ProspectNet outperforms the Cartesian product of two marginal predictions, and achieves comparable performance on the Interactive Motion Prediction benchmarks.
arXiv Detail & Related papers (2022-08-29T19:29:49Z) - Dynamic-Group-Aware Networks for Multi-Agent Trajectory Prediction with
Relational Reasoning [29.294244001911242]
We propose DynGroupNet, a dynamic-group-aware network, which can model time-varying interactions in highly dynamic scenes.
Based on DynGroupNet, we design a prediction system to forecast socially plausible trajectories with dynamic relational reasoning.
arXiv Detail & Related papers (2022-06-27T08:36:56Z) - Conversation Group Detection With Spatio-Temporal Context [11.288403109735544]
We propose an approach for detecting conversation groups in social scenarios like cocktail parties and networking events.
We posit the detection of conversation groups as a learning problem that could benefit from leveraging the spatial context of the surroundings.
This motivates our approach which consists of a dynamic LSTM-based deep learning model that predicts continuous pairwise affinity values.
arXiv Detail & Related papers (2022-06-02T08:05:02Z) - GroupNet: Multiscale Hypergraph Neural Networks for Trajectory
Prediction with Relational Reasoning [37.64048110462638]
GroupNet is a multiscale hypergraph neural network that captures both pair-wise and group-wise interactions.
We apply GroupNet into both CVAE-based prediction system and previous state-of-the-art prediction systems.
We show that with GroupNet, the CVAE-based prediction system outperforms state-of-the-art methods.
arXiv Detail & Related papers (2022-04-19T09:36:20Z) - Self-Regulated Learning for Egocentric Video Activity Anticipation [147.9783215348252]
Self-Regulated Learning (SRL) aims to regulate the intermediate representation consecutively to produce representation that emphasizes the novel information in the frame of the current time-stamp.
SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets.
arXiv Detail & Related papers (2021-11-23T03:29:18Z) - Long-Term Anticipation of Activities with Cycle Consistency [90.79357258104417]
We propose a framework for anticipating future activities directly from the features of the observed frames and train it in an end-to-end fashion.
Our framework achieves state-the-art results on two datasets: the Breakfast dataset and 50Salads.
arXiv Detail & Related papers (2020-09-02T15:41:32Z) - Learning to Abstract and Predict Human Actions [60.85905430007731]
We model the hierarchical structure of human activities in videos and demonstrate the power of such structure in action prediction.
We propose Hierarchical-Refresher-Anticipator, a multi-level neural machine that can learn the structure of human activities by observing a partial hierarchy of events and roll-out such structure into a future prediction in multiple levels of abstraction.
arXiv Detail & Related papers (2020-08-20T23:57:58Z) - Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
We propose an integrated solution based on the deep neural networks for temporal sets prediction.
A unique perspective is to learn element relationship by constructing set-level co-occurrence graph.
We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
arXiv Detail & Related papers (2020-06-20T03:29:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.