Classification Matters: Improving Video Action Detection with Class-Specific Attention

• URL: http://arxiv.org/abs/2407.19698v4
• Date: Wed, 11 Sep 2024 06:56:55 GMT
• Title: Classification Matters: Improving Video Action Detection with Class-Specific Attention
• Authors: Jinsung Lee, Taeoh Kim, Inwoong Lee, Minho Shim, Dongyoon Wee, Minsu Cho, Suha Kwak,
• Abstract summary: Video action detection (VAD) aims to detect actors and classify their actions in a video. We analyze how prevailing methods form features for classification and find that they prioritize actor regions. We propose to reduce the bias toward actor and encourage paying attention to the context that is relevant to each action class.
• Score: 61.14469113965433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video action detection (VAD) aims to detect actors and classify their actions in a video. We figure that VAD suffers more from classification rather than localization of actors. Hence, we analyze how prevailing methods form features for classification and find that they prioritize actor regions, yet often overlooking the essential contextual information necessary for accurate classification. Accordingly, we propose to reduce the bias toward actor and encourage paying attention to the context that is relevant to each action class. By assigning a class-dedicated query to each action class, our model can dynamically determine where to focus for effective classification. The proposed model demonstrates superior performance on three challenging benchmarks with significantly fewer parameters and less computation.

Related papers

• A Comprehensive Review of Few-shot Action Recognition [64.47305887411275]
  Few-shot action recognition aims to address the high cost and impracticality of manually labeling complex and variable video data. It requires accurately classifying human actions in videos using only a few labeled examples per class.
  arXiv  Detail & Related papers  (2024-07-20T03:53:32Z)
• Open-Vocabulary Spatio-Temporal Action Detection [59.91046192096296]
  Open-vocabulary-temporal action detection (OV-STAD) is an important fine-grained video understanding task. OV-STAD requires training a model on a limited set of base classes with box and label supervision. To better adapt the holistic VLM for the fine-grained action detection task, we carefully fine-tune it on the localized video region-text pairs.
  arXiv  Detail & Related papers  (2024-05-17T14:52:47Z)
• Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling [19.683714649646603]
  Weakly-supervised action localization aims to recognize and localize action instancese in untrimmed videos with only video-level labels. Most existing models rely on multiple instance learning(MIL), where predictions of unlabeled instances are supervised by classifying labeled bags. We propose a novel attention-based hierarchically-structured latent model to learn the temporal variations of feature semantics.
  arXiv  Detail & Related papers  (2023-08-19T08:45:49Z)
• Multi-modal Prompting for Low-Shot Temporal Action Localization [95.19505874963751]
  We consider the problem of temporal action localization under low-shot (zero-shot & few-shot) scenario. We adopt a Transformer-based two-stage action localization architecture with class-agnostic action proposal, followed by open-vocabulary classification.
  arXiv  Detail & Related papers  (2023-03-21T10:40:13Z)
• Fine-Grained Visual Classification using Self Assessment Classifier [12.596520707449027]
  Extracting discriminative features plays a crucial role in the fine-grained visual classification task. In this paper, we introduce a Self Assessment, which simultaneously leverages the representation of the image and top-k prediction classes. We show that our method achieves new state-of-the-art results on CUB200-2011, Stanford Dog, and FGVC Aircraft datasets.
  arXiv  Detail & Related papers  (2022-05-21T07:41:27Z)
• Self-supervised Pretraining with Classification Labels for Temporal Activity Detection [54.366236719520565]
  Temporal Activity Detection aims to predict activity classes per frame. Due to the expensive frame-level annotations required for detection, the scale of detection datasets is limited. This work proposes a novel self-supervised pretraining method for detection leveraging classification labels.
  arXiv  Detail & Related papers  (2021-11-26T18:59:28Z)
• TNT: Text-Conditioned Network with Transductive Inference for Few-Shot Video Classification [26.12591949900602]
  We formulate a text-based task conditioner to adapt video features to the few-shot learning task. Our model obtains state-of-the-art performance on four challenging benchmarks in few-shot video action classification.
  arXiv  Detail & Related papers  (2021-06-21T15:08:08Z)
• Exploiting stance hierarchies for cost-sensitive stance detection of Web documents [24.898077978955406]
  stance detection aims at identifying the position (stance) of a document towards a claim. We propose a modular pipeline of cascading binary classifiers, enabling performance tuning on a per step and class basis. We implement our approach through a combination of neural and traditional classification models that highlight the misclassification costs of minority classes.
  arXiv  Detail & Related papers  (2020-07-29T21:40:01Z)
• Fine-Grained Visual Classification with Efficient End-to-end Localization [49.9887676289364]
  We present an efficient localization module that can be fused with a classification network in an end-to-end setup. We evaluate the new model on the three benchmark datasets CUB200-2011, Stanford Cars and FGVC-Aircraft.
  arXiv  Detail & Related papers  (2020-05-11T14:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.