Energy-based Periodicity Mining with Deep Features for Action Repetition Counting in Unconstrained Videos

• URL: http://arxiv.org/abs/2003.06838v1
• Date: Sun, 15 Mar 2020 14:21:18 GMT
• Title: Energy-based Periodicity Mining with Deep Features for Action Repetition Counting in Unconstrained Videos
• Authors: Jianqin Yin and Yanchun Wu and Huaping Liu and Yonghao Dang and Zhiyi Liu and Jun Liu
• Abstract summary: Action repetition counting is to estimate the occurrence times of the repetitive motion in one action. We propose a new method superior to the traditional ways in two aspects, without preprocessing and applicable for arbitrary periodicity actions.
• Score: 17.00863997561408
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Action repetition counting is to estimate the occurrence times of the repetitive motion in one action, which is a relatively new, important but challenging measurement problem. To solve this problem, we propose a new method superior to the traditional ways in two aspects, without preprocessing and applicable for arbitrary periodicity actions. Without preprocessing, the proposed model makes our method convenient for real applications; processing the arbitrary periodicity action makes our model more suitable for the actual circumstance. In terms of methodology, firstly, we analyze the movement patterns of the repetitive actions based on the spatial and temporal features of actions extracted by deep ConvNets; Secondly, the Principal Component Analysis algorithm is used to generate the intuitive periodic information from the chaotic high-dimensional deep features; Thirdly, the periodicity is mined based on the high-energy rule using Fourier transform; Finally, the inverse Fourier transform with a multi-stage threshold filter is proposed to improve the quality of the mined periodicity, and peak detection is introduced to finish the repetition counting. Our work features two-fold: 1) An important insight that deep features extracted for action recognition can well model the self-similarity periodicity of the repetitive action is presented. 2) A high-energy based periodicity mining rule using deep features is presented, which can process arbitrary actions without preprocessing. Experimental results show that our method achieves comparable results on the public datasets YT Segments and QUVA.

Related papers

• Efficient Action Counting with Dynamic Queries [31.833468477101604]
  We introduce a novel approach that employs an action query representation to localize repeated action cycles with linear computational complexity. Unlike static action queries, this approach dynamically embeds video features into action queries, offering a more flexible and generalizable representation. Our method significantly outperforms previous works, particularly in terms of long video sequences, unseen actions, and actions at various speeds.
  arXiv  Detail & Related papers  (2024-03-03T15:43:11Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• GaitFormer: Revisiting Intrinsic Periodicity for Gait Recognition [6.517046095186713]
  Gait recognition aims to distinguish different walking patterns by analyzing video-level human silhouettes, rather than relying on appearance information. Previous research has primarily focused on extracting local or global-temporal representations, while overlooking the intrinsic periodic features of gait sequences. We propose a plug-and-play strategy, called Temporal Periodic Alignment (TPA), which leverages the periodic nature and fine-grained temporal dependencies of gait patterns.
  arXiv  Detail & Related papers  (2023-07-25T05:05:07Z)
• Diffusion Action Segmentation [63.061058214427085]
  We propose a novel framework via denoising diffusion models, which shares the same inherent spirit of such iterative refinement. In this framework, action predictions are iteratively generated from random noise with input video features as conditions.
  arXiv  Detail & Related papers  (2023-03-31T10:53:24Z)
• Spatio-Temporal Point Process for Multiple Object Tracking [30.041104276095624]
  Multiple Object Tracking (MOT) focuses on modeling the relationship of detected objects among consecutive frames and merge them into different trajectories. We propose a novel framework that can effectively predict and mask-out noisy and confusing detection results before associating objects into trajectories.
  arXiv  Detail & Related papers  (2023-02-05T18:14:08Z)
• Time Series Anomaly Detection by Cumulative Radon Features [32.36217153362305]
  In this work, we argue that shallow features suffice when combined with distribution distance measures. Our approach models each time series as a high dimensional empirical distribution of features, where each time-point constitutes a single sample. We show that by parameterizing each time series using cumulative Radon features, we are able to efficiently and effectively model the distribution of normal time series.
  arXiv  Detail & Related papers  (2022-02-08T18:58:53Z)
• Sequence-to-Sequence Modeling for Action Identification at High Temporal Resolution [9.902223920743872]
  We introduce a new action-recognition benchmark that includes subtle short-duration actions labeled at a high temporal resolution. We show that current state-of-the-art models based on segmentation produce noisy predictions when applied to these data. We propose a novel approach for high-resolution action identification, inspired by speech-recognition techniques.
  arXiv  Detail & Related papers  (2021-11-03T21:06:36Z)
• On Contrastive Representations of Stochastic Processes [53.21653429290478]
  Learning representations of processes is an emerging problem in machine learning. We show that our methods are effective for learning representations of periodic functions, 3D objects and dynamical processes.
  arXiv  Detail & Related papers  (2021-06-18T11:00:24Z)
• Learning Salient Boundary Feature for Anchor-free Temporal Action Localization [81.55295042558409]
  Temporal action localization is an important yet challenging task in video understanding. We propose the first purely anchor-free temporal localization method. Our model includes (i) an end-to-end trainable basic predictor, (ii) a saliency-based refinement module, and (iii) several consistency constraints.
  arXiv  Detail & Related papers  (2021-03-24T12:28:32Z)
• Collaborative Distillation in the Parameter and Spectrum Domains for Video Action Recognition [79.60708268515293]
  This paper explores how to train small and efficient networks for action recognition. We propose two distillation strategies in the frequency domain, namely the feature spectrum and parameter distribution distillations respectively. Our method can achieve higher performance than state-of-the-art methods with the same backbone.
  arXiv  Detail & Related papers  (2020-09-15T07:29:57Z)
• Context-aware and Scale-insensitive Temporal Repetition Counting [60.40438811580856]
  Temporal repetition counting aims to estimate the number of cycles of a given repetitive action. Existing deep learning methods assume repetitive actions are performed in a fixed time-scale, which is invalid for the complex repetitive actions in real life. We propose a context-aware and scale-insensitive framework to tackle the challenges in repetition counting caused by the unknown and diverse cycle-lengths.
  arXiv  Detail & Related papers  (2020-05-18T05:49:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.