Continual Action Assessment via Task-Consistent Score-Discriminative Feature Distribution Modeling

• URL: http://arxiv.org/abs/2309.17105v5
• Date: Thu, 2 May 2024 08:44:23 GMT
• Title: Continual Action Assessment via Task-Consistent Score-Discriminative Feature Distribution Modeling
• Authors: Yuan-Ming Li, Ling-An Zeng, Jing-Ke Meng, Wei-Shi Zheng,
• Abstract summary: Action Quality Assessment (AQA) is a task that tries to answer how well an action is carried out. Existing works on AQA assume that all the training data are visible for training at one time, but do not enable continual learning. We propose a unified model to learn AQA tasks sequentially without forgetting.
• Score: 31.696222064667243
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Action Quality Assessment (AQA) is a task that tries to answer how well an action is carried out. While remarkable progress has been achieved, existing works on AQA assume that all the training data are visible for training at one time, but do not enable continual learning on assessing new technical actions. In this work, we address such a Continual Learning problem in AQA (Continual-AQA), which urges a unified model to learn AQA tasks sequentially without forgetting. Our idea for modeling Continual-AQA is to sequentially learn a task-consistent score-discriminative feature distribution, in which the latent features express a strong correlation with the score labels regardless of the task or action types.From this perspective, we aim to mitigate the forgetting in Continual-AQA from two aspects. Firstly, to fuse the features of new and previous data into a score-discriminative distribution, a novel Feature-Score Correlation-Aware Rehearsal is proposed to store and reuse data from previous tasks with limited memory size. Secondly, an Action General-Specific Graph is developed to learn and decouple the action-general and action-specific knowledge so that the task-consistent score-discriminative features can be better extracted across various tasks. Extensive experiments are conducted to evaluate the contributions of proposed components. The comparisons with the existing continual learning methods additionally verify the effectiveness and versatility of our approach. Data and code are available at https://github.com/iSEE-Laboratory/Continual-AQA.

Related papers

• UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment [23.48816491333345]
  Image Quality Assessment (IQA) and Image Aesthetic Assessment (IAA) aim to simulate human subjective perception of image visual quality and aesthetic appeal. Existing methods typically address these tasks independently due to distinct learning objectives. We propose Unified vision-language pre-training of Quality and Aesthetics (UniQA) to learn general perceptions of two tasks, thereby benefiting them simultaneously.
  arXiv  Detail & Related papers  (2024-06-03T07:40:10Z)
• Adaptive Rentention & Correction for Continual Learning [114.5656325514408]
  A common problem in continual learning is the classification layer's bias towards the most recent task. We name our approach Adaptive Retention & Correction (ARC) ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets.
  arXiv  Detail & Related papers  (2024-05-23T08:43:09Z)
• CoFInAl: Enhancing Action Quality Assessment with Coarse-to-Fine Instruction Alignment [38.12600984070689]
  Action Quality Assessment (AQA) is pivotal for quantifying actions across domains like sports and medical care. Existing methods often rely on pre-trained backbones from large-scale action recognition datasets to boost performance on smaller AQA datasets. We propose Coarse-to-Fine Instruction Alignment (CoFInAl) to align AQA with broader pre-trained tasks by reformulating it as a coarse-to-fine classification task.
  arXiv  Detail & Related papers  (2024-04-22T09:03:21Z)
• Multi-Stage Contrastive Regression for Action Quality Assessment [31.763380011104015]
  We propose a novel Multi-stage Contrastive Regression (MCoRe) framework for the action quality assessment (AQA) task. Inspired by the graph contrastive learning, we propose a new stage-wise contrastive learning loss function to enhance performance. MCoRe demonstrates the state-of-the-art result so far on the widely-adopted fine-grained AQA dataset.
  arXiv  Detail & Related papers  (2024-01-05T14:48:19Z)
• Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training [106.85813323510783]
  This paper presents a novel paradigm for the Zero-Shot Item-based Recommendation (ZSIR) task. It pre-trains a model on product knowledge graph (PKG) to refine the item features from PLMs. We identify three challenges for pre-training PKG, which are multi-type relations in PKG, semantic divergence between item generic information and relations and domain discrepancy from PKG to downstream ZSIR task.
  arXiv  Detail & Related papers  (2023-05-12T17:38:24Z)
• Composite Learning for Robust and Effective Dense Predictions [81.2055761433725]
  Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task. We find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
  arXiv  Detail & Related papers  (2022-10-13T17:59:16Z)
• Forgetting to Remember: A Scalable Incremental Learning Framework for Cross-Task Blind Image Quality Assessment [25.67247922033185]
  This paper proposes a scalable incremental learning framework (SILF) that could sequentially conduct blind image quality assessment (BIQA) across multiple evaluation tasks with limited memory capacity. To suppress the unrestrained expansion of memory capacity in sequential learning, we develop a scalable memory unit by gradually and selectively pruning unimportant neurons from previously settled parameter subsets.
  arXiv  Detail & Related papers  (2022-09-15T08:19:12Z)
• Identifying Auxiliary or Adversarial Tasks Using Necessary Condition Analysis for Adversarial Multi-task Video Understanding [34.75145779372538]
  We propose a generalized notion of multi-task learning by incorporating both auxiliary tasks that the model should perform well on and adversarial tasks that the model should not perform well on. Our novel proposed framework, Adversarial Multi-Task Neural Networks (AMT), penalizes adversarial tasks, determined by NCA to be scene recognition. We show that our approach improves accuracy by 3% and encourages the model to attend to action features instead of correlation-biasing scene features.
  arXiv  Detail & Related papers  (2022-08-22T06:26:11Z)
• ReAct: Temporal Action Detection with Relational Queries [84.76646044604055]
  This work aims at advancing temporal action detection (TAD) using an encoder-decoder framework with action queries. We first propose a relational attention mechanism in the decoder, which guides the attention among queries based on their relations. Lastly, we propose to predict the localization quality of each action query at inference in order to distinguish high-quality queries.
  arXiv  Detail & Related papers  (2022-07-14T17:46:37Z)
• Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism [120.1998866178014]
  We present a flexible framework for continual object detection via pRotOtypical taSk corrElaTion guided gaTingAnism (ROSETTA) Concretely, a unified framework is shared by all tasks while task-aware gates are introduced to automatically select sub-models for specific tasks. Experiments on COCO-VOC, KITTI-Kitchen, class-incremental detection on VOC and sequential learning of four tasks show that ROSETTA yields state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-06T07:31:28Z)
• Harvesting and Refining Question-Answer Pairs for Unsupervised QA [95.9105154311491]
  We introduce two approaches to improve unsupervised Question Answering (QA) First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as RefQA) Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over RefQA.
  arXiv  Detail & Related papers  (2020-05-06T15:56:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.