ActionSpotter: Deep Reinforcement Learning Framework for Temporal Action Spotting in Videos

• URL: http://arxiv.org/abs/2004.06971v2
• Date: Tue, 10 Nov 2020 16:43:56 GMT
• Title: ActionSpotter: Deep Reinforcement Learning Framework for Temporal Action Spotting in Videos
• Authors: Guillaume Vaudaux-Ruth, Adrien Chan-Hon-Tong, Catherine Achard (ISIR, PIROS, SU)
• Abstract summary: ActionSpotter is a spotting algorithm that takes advantage of Deep Reinforcement Learning to efficiently spot actions while adapting its video browsing speed. In particular, the spotting mean Average Precision on THUMOS14 is significantly improved from 59.7% to 65.6% while skipping 23% of video.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Summarizing video content is an important task in many applications. This task can be defined as the computation of the ordered list of actions present in a video. Such a list could be extracted using action detection algorithms. However, it is not necessary to determine the temporal boundaries of actions to know their existence. Moreover, localizing precise boundaries usually requires dense video analysis to be effective. In this work, we propose to directly compute this ordered list by sparsely browsing the video and selecting one frame per action instance, task known as action spotting in literature. To do this, we propose ActionSpotter, a spotting algorithm that takes advantage of Deep Reinforcement Learning to efficiently spot actions while adapting its video browsing speed, without additional supervision. Experiments performed on datasets THUMOS14 and ActivityNet show that our framework outperforms state of the art detection methods. In particular, the spotting mean Average Precision on THUMOS14 is significantly improved from 59.7% to 65.6% while skipping 23% of video.

Related papers

• FMI-TAL: Few-shot Multiple Instances Temporal Action Localization by Probability Distribution Learning and Interval Cluster Refinement [2.261014973523156]
  We propose a novel solution involving a spatial-channel relation transformer with probability learning and cluster refinement. This method can accurately identify the start and end boundaries of actions in the query video. Our model achieves competitive performance through meticulous experimentation utilizing the benchmark datasets ActivityNet1.3 and THUMOS14.
  arXiv  Detail & Related papers  (2024-08-25T08:17:25Z)
• Boundary-Denoising for Video Activity Localization [57.9973253014712]
  We study the video activity localization problem from a denoising perspective. Specifically, we propose an encoder-decoder model named DenoiseLoc. Experiments show that DenoiseLoc advances %in several video activity understanding tasks.
  arXiv  Detail & Related papers  (2023-04-06T08:48:01Z)
• ETAD: A Unified Framework for Efficient Temporal Action Detection [70.21104995731085]
  Untrimmed video understanding such as temporal action detection (TAD) often suffers from the pain of huge demand for computing resources. We build a unified framework for efficient end-to-end temporal action detection (ETAD) ETAD achieves state-of-the-art performance on both THUMOS-14 and ActivityNet-1.3.
  arXiv  Detail & Related papers  (2022-05-14T21:16:21Z)
• Video Action Detection: Analysing Limitations and Challenges [70.01260415234127]
  We analyze existing datasets on video action detection and discuss their limitations. We perform a biasness study which analyzes a key property differentiating videos from static images: the temporal aspect. Such extreme experiments show existence of biases which have managed to creep into existing methods inspite of careful modeling.
  arXiv  Detail & Related papers  (2022-04-17T00:42:14Z)
• Deep Learning-based Action Detection in Untrimmed Videos: A Survey [20.11911785578534]
  Most real-world videos are lengthy and untrimmed with sparse segments of interest. The task of temporal activity detection in untrimmed videos aims to localize the temporal boundary of actions. This paper provides an overview of deep learning-based algorithms to tackle temporal action detection in untrimmed videos.
  arXiv  Detail & Related papers  (2021-09-30T22:42:25Z)
• TinyVIRAT: Low-resolution Video Action Recognition [70.37277191524755]
  In real-world surveillance environments, the actions in videos are captured at a wide range of resolutions. We introduce a benchmark dataset, TinyVIRAT, which contains natural low-resolution activities. We propose a novel method for recognizing tiny actions in videos which utilizes a progressive generative approach.
  arXiv  Detail & Related papers  (2020-07-14T21:09:18Z)
• Gabriella: An Online System for Real-Time Activity Detection in Untrimmed Security Videos [72.50607929306058]
  We propose a real-time online system to perform activity detection on untrimmed security videos. The proposed method consists of three stages: tubelet extraction, activity classification and online tubelet merging. We demonstrate the effectiveness of the proposed approach in terms of speed (100 fps) and performance with state-of-the-art results.
  arXiv  Detail & Related papers  (2020-04-23T22:20:10Z)
• ZSTAD: Zero-Shot Temporal Activity Detection [107.63759089583382]
  We propose a novel task setting called zero-shot temporal activity detection (ZSTAD), where activities that have never been seen in training can still be detected. We design an end-to-end deep network based on R-C3D as the architecture for this solution. Experiments on both the THUMOS14 and the Charades datasets show promising performance in terms of detecting unseen activities.
  arXiv  Detail & Related papers  (2020-03-12T02:40:36Z)
• Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequences [25.299599341774204]
  This paper proposes an approach for the unsupervised learning of actions in untrimmed video sequences based on a joint visual-temporal embedding space. We show that the proposed approach is able to provide a meaningful visual and temporal embedding out of the visual cues present in contiguous video frames.
  arXiv  Detail & Related papers  (2020-01-29T22:51:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.