ProActive: Self-Attentive Temporal Point Process Flows for Activity Sequences

• URL: http://arxiv.org/abs/2206.05291v1
• Date: Fri, 10 Jun 2022 16:30:55 GMT
• Title: ProActive: Self-Attentive Temporal Point Process Flows for Activity Sequences
• Authors: Vinayak Gupta and Srikanta Bedathur
• Abstract summary: ProActive is a framework for modeling the continuous-time distribution of actions in an activity sequence. It addresses next action prediction, sequence-goal prediction, and end-to-end sequence generation.
• Score: 9.571588145356277
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Any human activity can be represented as a temporal sequence of actions performed to achieve a certain goal. Unlike machine-made time series, these action sequences are highly disparate as the time taken to finish a similar action might vary between different persons. Therefore, understanding the dynamics of these sequences is essential for many downstream tasks such as activity length prediction, goal prediction, etc. Existing neural approaches that model an activity sequence are either limited to visual data or are task specific, i.e., limited to next action or goal prediction. In this paper, we present ProActive, a neural marked temporal point process (MTPP) framework for modeling the continuous-time distribution of actions in an activity sequence while simultaneously addressing three high-impact problems -- next action prediction, sequence-goal prediction, and end-to-end sequence generation. Specifically, we utilize a self-attention module with temporal normalizing flows to model the influence and the inter-arrival times between actions in a sequence. Moreover, for time-sensitive prediction, we perform an early detection of sequence goal via a constrained margin-based optimization procedure. This in-turn allows ProActive to predict the sequence goal using a limited number of actions. Extensive experiments on sequences derived from three activity recognition datasets show the significant accuracy boost of ProActive over the state-of-the-art in terms of action and goal prediction, and the first-ever application of end-to-end action sequence generation.

Related papers

• Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction [68.90783662117936]
  Click-through Rate (CTR) prediction is crucial for online personalization platforms. Recent advancements have shown that modeling rich user behaviors can significantly improve the performance of CTR prediction. We propose Multi-granularity Interest Retrieval and Refinement Network (MIRRN)
  arXiv  Detail & Related papers  (2024-11-22T15:29:05Z)
• Finding the DeepDream for Time Series: Activation Maximization for Univariate Time Series [10.388704631887496]
  We introduce Sequence Dreaming, a technique that adapts Maxim Activationization to analyze sequential information. We visualize the temporal dynamics and patterns most influential in model decision-making processes.
  arXiv  Detail & Related papers  (2024-08-20T08:09:44Z)
• Meta-Learning for Neural Network-based Temporal Point Processes [36.31950058651308]
  The point process is widely used to predict events related to human activities. Recent high-performance point process models require the input of sufficient numbers of events collected over a long period. We propose a novel meta-learning approach for periodicity-aware prediction of future events given short sequences.
  arXiv  Detail & Related papers  (2024-01-29T02:42:22Z)
• Activity Grammars for Temporal Action Segmentation [71.03141719666972]
  temporal action segmentation aims at translating an untrimmed activity video into a sequence of action segments. This paper introduces an effective activity grammar to guide neural predictions for temporal action segmentation. Experimental results demonstrate that our method significantly improves temporal action segmentation in terms of both performance and interpretability.
  arXiv  Detail & Related papers  (2023-12-07T12:45:33Z)
• Tapestry of Time and Actions: Modeling Human Activity Sequences using Temporal Point Process Flows [9.571588145356277]
  We present ProActive, a framework for modeling the continuous-time distribution of actions in an activity sequence. ProActive addresses three high-impact problems -- next action prediction, sequence-goal prediction, and end-to-end sequence generation.
  arXiv  Detail & Related papers  (2023-07-13T19:17:54Z)
• Finding Islands of Predictability in Action Forecasting [7.215559809521136]
  We show that future action sequences are more accurately modeled with variable, rather than one, levels of abstraction. We propose a combination Bayesian neural network and hierarchical convolutional segmentation model to both accurately predict future actions and optimally select abstraction levels.
  arXiv  Detail & Related papers  (2022-10-13T21:01:16Z)
• Learning Sequence Representations by Non-local Recurrent Neural Memory [61.65105481899744]
  We propose a Non-local Recurrent Neural Memory (NRNM) for supervised sequence representation learning. Our model is able to capture long-range dependencies and latent high-level features can be distilled by our model. Our model compares favorably against other state-of-the-art methods specifically designed for each of these sequence applications.
  arXiv  Detail & Related papers  (2022-07-20T07:26:15Z)
• The Wisdom of Crowds: Temporal Progressive Attention for Early Action Prediction [104.628661890361]
  Early action prediction deals with inferring the ongoing action from partially-observed videos, typically at the outset of the video. We propose a bottleneck-based attention model that captures the evolution of the action, through progressive sampling over fine-to-coarse scales.
  arXiv  Detail & Related papers  (2022-04-28T08:21:09Z)
• Sequence-to-Sequence Modeling for Action Identification at High Temporal Resolution [9.902223920743872]
  We introduce a new action-recognition benchmark that includes subtle short-duration actions labeled at a high temporal resolution. We show that current state-of-the-art models based on segmentation produce noisy predictions when applied to these data. We propose a novel approach for high-resolution action identification, inspired by speech-recognition techniques.
  arXiv  Detail & Related papers  (2021-11-03T21:06:36Z)
• A Prospective Study on Sequence-Driven Temporal Sampling and Ego-Motion Compensation for Action Recognition in the EPIC-Kitchens Dataset [68.8204255655161]
  Action recognition is one of the top-challenging research fields in computer vision. ego-motion recorded sequences have become of important relevance. The proposed method aims to cope with it by estimating this ego-motion or camera motion.
  arXiv  Detail & Related papers  (2020-08-26T14:44:45Z)
• MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation [87.16030562892537]
  We propose a multi-stage architecture for the temporal action segmentation task. The first stage generates an initial prediction that is refined by the next ones. Our models achieve state-of-the-art results on three datasets.
  arXiv  Detail & Related papers  (2020-06-16T14:50:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.