Group-aware Contrastive Regression for Action Quality Assessment

• URL: http://arxiv.org/abs/2108.07797v1
• Date: Tue, 17 Aug 2021 17:59:39 GMT
• Title: Group-aware Contrastive Regression for Action Quality Assessment
• Authors: Xumin Yu, Yongming Rao, Wenliang Zhao, Jiwen Lu, Jie Zhou
• Abstract summary: We show that the relations among videos can provide important clues for more accurate action quality assessment. Our approach outperforms previous methods by a large margin and establishes new state-of-the-art on all three benchmarks.
• Score: 85.43203180953076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Assessing action quality is challenging due to the subtle differences between videos and large variations in scores. Most existing approaches tackle this problem by regressing a quality score from a single video, suffering a lot from the large inter-video score variations. In this paper, we show that the relations among videos can provide important clues for more accurate action quality assessment during both training and inference. Specifically, we reformulate the problem of action quality assessment as regressing the relative scores with reference to another video that has shared attributes (e.g., category and difficulty), instead of learning unreferenced scores. Following this formulation, we propose a new Contrastive Regression (CoRe) framework to learn the relative scores by pair-wise comparison, which highlights the differences between videos and guides the models to learn the key hints for assessment. In order to further exploit the relative information between two videos, we devise a group-aware regression tree to convert the conventional score regression into two easier sub-problems: coarse-to-fine classification and regression in small intervals. To demonstrate the effectiveness of CoRe, we conduct extensive experiments on three mainstream AQA datasets including AQA-7, MTL-AQA and JIGSAWS. Our approach outperforms previous methods by a large margin and establishes new state-of-the-art on all three benchmarks.

Related papers

• Not All Pairs are Equal: Hierarchical Learning for Average-Precision-Oriented Video Retrieval [80.09819072780193]
  Average Precision (AP) assesses the overall rankings of relevant videos at the top list. Recent video retrieval methods utilize pair-wise losses that treat all sample pairs equally.
  arXiv  Detail & Related papers  (2024-07-22T11:52:04Z)
• Multi-Stage Contrastive Regression for Action Quality Assessment [31.763380011104015]
  We propose a novel Multi-stage Contrastive Regression (MCoRe) framework for the action quality assessment (AQA) task. Inspired by the graph contrastive learning, we propose a new stage-wise contrastive learning loss function to enhance performance. MCoRe demonstrates the state-of-the-art result so far on the widely-adopted fine-grained AQA dataset.
  arXiv  Detail & Related papers  (2024-01-05T14:48:19Z)
• CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
  Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
  arXiv  Detail & Related papers  (2022-06-29T15:22:01Z)
• Auto-Encoding Score Distribution Regression for Action Quality Assessment [41.45638722765149]
  Action quality assessment (AQA) from videos is a challenging vision task. Traditionally, AQA task is treated as a regression problem to learn the underlying mappings between videos and action scores. We develop Distribution Auto-Encoder (DAE) to address the above problems.
  arXiv  Detail & Related papers  (2021-11-22T07:30:04Z)
• CoCon: Cooperative-Contrastive Learning [52.342936645996765]
  Self-supervised visual representation learning is key for efficient video analysis. Recent success in learning image representations suggests contrastive learning is a promising framework to tackle this challenge. We introduce a cooperative variant of contrastive learning to utilize complementary information across views.
  arXiv  Detail & Related papers  (2021-04-30T05:46:02Z)
• Regression or Classification? New Methods to Evaluate No-Reference Picture and Video Quality Models [45.974399400141685]
  We propose two new methods to evaluate and compare no-reference quality models at coarser levels. We conduct a benchmark experiment of popular no-reference quality models on recent in-the-wild picture and video quality datasets.
  arXiv  Detail & Related papers  (2021-01-30T05:40:14Z)
• Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation [132.82884193921535]
  We argue that previous methods underestimate the importance of video feature learning and propose a two-stage approach. We show that this simple baseline approach outperforms prior few-shot video classification methods by over 20 points on existing benchmarks. We present two novel approaches that yield further improvement.
  arXiv  Detail & Related papers  (2020-07-09T13:05:32Z)
• Uncertainty-aware Score Distribution Learning for Action Quality Assessment [91.05846506274881]
  We propose an uncertainty-aware score distribution learning (USDL) approach for action quality assessment (AQA) Specifically, we regard an action as an instance associated with a score distribution, which describes the probability of different evaluated scores. Under the circumstance where fine-grained score labels are available, we devise a multi-path uncertainty-aware score distributions learning (MUSDL) method to explore the disentangled components of a score.
  arXiv  Detail & Related papers  (2020-06-13T15:41:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.