Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks

• URL: http://arxiv.org/abs/2002.01449v1
• Date: Tue, 4 Feb 2020 18:21:10 GMT
• Title: Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks
• Authors: Maheen Rashid, Hedvig Kjellstr\"om, Yong Jae Lee
• Abstract summary: We present a method for weakly-supervised action localization based on graph convolutions. Our method utilizes similarity graphs that encode appearance and motion, and pushes the state of the art on THUMOS '14, ActivityNet 1.2, and Charades for weakly supervised action localization.
• Score: 25.342482374259017
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a method for weakly-supervised action localization based on graph convolutions. In order to find and classify video time segments that correspond to relevant action classes, a system must be able to both identify discriminative time segments in each video, and identify the full extent of each action. Achieving this with weak video level labels requires the system to use similarity and dissimilarity between moments across videos in the training data to understand both how an action appears, as well as the sub-actions that comprise the action's full extent. However, current methods do not make explicit use of similarity between video moments to inform the localization and classification predictions. We present a novel method that uses graph convolutions to explicitly model similarity between video moments. Our method utilizes similarity graphs that encode appearance and motion, and pushes the state of the art on THUMOS '14, ActivityNet 1.2, and Charades for weakly supervised action localization.

Related papers

• Free-Form Composition Networks for Egocentric Action Recognition [97.02439848145359]
  We propose a free-form composition network (FFCN) that can simultaneously learn disentangled verb, preposition, and noun representations. The proposed FFCN can directly generate new training data samples for rare classes, hence significantly improve action recognition performance.
  arXiv  Detail & Related papers  (2023-07-13T02:22:09Z)
• Video alignment using unsupervised learning of local and global features [0.0]
  We introduce an unsupervised method for alignment that uses global and local features of the frames. In particular, we introduce effective features for each video frame by means of three machine vision tools: person detection, pose estimation, and VGG network. The main advantage of our approach is that no training is required, which makes it applicable for any new type of action without any need to collect training samples for it.
  arXiv  Detail & Related papers  (2023-04-13T22:20:54Z)
• Representing Videos as Discriminative Sub-graphs for Action Recognition [165.54738402505194]
  We introduce a new design of sub-graphs to represent and encode theriminative patterns of each action in the videos. We present MUlti-scale Sub-Earn Ling (MUSLE) framework that novelly builds space-time graphs and clusters into compact sub-graphs on each scale.
  arXiv  Detail & Related papers  (2022-01-11T16:15:25Z)
• Graph Convolutional Module for Temporal Action Localization in Videos [142.5947904572949]
  We claim that the relations between action units play an important role in action localization. A more powerful action detector should not only capture the local content of each action unit but also allow a wider field of view on the context related to it. We propose a general graph convolutional module (GCM) that can be easily plugged into existing action localization methods.
  arXiv  Detail & Related papers  (2021-12-01T06:36:59Z)
• Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation [79.50868197788773]
  We develop a more accurate weakly-supervised solution by introducing Cross-Sentence Relations Mining. We explore two cross-sentence relational constraints: (1) trimmed ordering and (2) semantic consistency among sentences in a paragraph description of video activities. Experiments on two publicly available activity localisation datasets show the advantages of our approach over the state-of-the-art weakly supervised methods.
  arXiv  Detail & Related papers  (2021-07-23T20:04:01Z)
• Improved Actor Relation Graph based Group Activity Recognition [0.0]
  The detailed description of human actions and group activities is essential information, which can be used in real-time CCTV video surveillance, health care, sports video analysis, etc. This study proposes a video understanding method that mainly focused on group activity recognition by learning the pair-wise actor appearance similarity and actor positions.
  arXiv  Detail & Related papers  (2020-10-24T19:46:49Z)
• Learning to Localize Actions from Moments [153.54638582696128]
  We introduce a new design of transfer learning type to learn action localization for a large set of action categories. We present Action Herald Networks (AherNet) that integrate such design into an one-stage action localization framework.
  arXiv  Detail & Related papers  (2020-08-31T16:03:47Z)
• SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentation [22.887397951846353]
  Weakly supervised approaches aim at learning temporal action segmentation from videos that are only weakly labeled. We propose an approach that can be trained end-to-end on such data. We evaluate our approach on three datasets where the approach achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2020-03-31T14:51:41Z)
• Weakly-Supervised Action Localization by Generative Attention Modeling [65.03548422403061]
  Weakly-supervised temporal action localization is a problem of learning an action localization model with only video-level action labeling available. We propose to model the class-agnostic frame-wise conditioned probability on the frame attention using conditional Variational Auto-Encoder (VAE) By maximizing the conditional probability with respect to the attention, the action and non-action frames are well separated.
  arXiv  Detail & Related papers  (2020-03-27T14:02:56Z)
• Weakly Supervised Temporal Action Localization Using Deep Metric Learning [12.49814373580862]
  We propose a weakly supervised temporal action localization method that only requires video-level action instances as supervision during training. We jointly optimize a balanced binary cross-entropy loss and a metric loss using a standard backpropagation algorithm. Our approach improves the current state-of-the-art result for THUMOS14 by 6.5% mAP at IoU threshold 0.5, and achieves competitive performance for ActivityNet1.2.
  arXiv  Detail & Related papers  (2020-01-21T22:01:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.