Enhancing Blind Video Quality Assessment with Rich Quality-aware Features

• URL: http://arxiv.org/abs/2405.08745v1
• Date: Tue, 14 May 2024 16:32:11 GMT
• Title: Enhancing Blind Video Quality Assessment with Rich Quality-aware Features
• Authors: Wei Sun, Haoning Wu, Zicheng Zhang, Jun Jia, Zhichao Zhang, Linhan Cao, Qiubo Chen, Xiongkuo Min, Weisi Lin, Guangtao Zhai,
• Abstract summary: We present a simple but effective method to enhance blind video quality assessment (BVQA) models for social media videos. We explore rich quality-aware features from pre-trained blind image quality assessment (BIQA) and BVQA models as auxiliary features. Experimental results demonstrate that the proposed model achieves the best performance on three public social media VQA datasets.
• Score: 79.18772373737724
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a simple but effective method to enhance blind video quality assessment (BVQA) models for social media videos. Motivated by previous researches that leverage pre-trained features extracted from various computer vision models as the feature representation for BVQA, we further explore rich quality-aware features from pre-trained blind image quality assessment (BIQA) and BVQA models as auxiliary features to help the BVQA model to handle complex distortions and diverse content of social media videos. Specifically, we use SimpleVQA, a BVQA model that consists of a trainable Swin Transformer-B and a fixed SlowFast, as our base model. The Swin Transformer-B and SlowFast components are responsible for extracting spatial and motion features, respectively. Then, we extract three kinds of features from Q-Align, LIQE, and FAST-VQA to capture frame-level quality-aware features, frame-level quality-aware along with scene-specific features, and spatiotemporal quality-aware features, respectively. Through concatenating these features, we employ a multi-layer perceptron (MLP) network to regress them into quality scores. Experimental results demonstrate that the proposed model achieves the best performance on three public social media VQA datasets. Moreover, the proposed model won first place in the CVPR NTIRE 2024 Short-form UGC Video Quality Assessment Challenge. The code is available at \url{https://github.com/sunwei925/RQ-VQA.git}.

Related papers

• VQA$^2$: Visual Question Answering for Video Quality Assessment [76.81110038738699]
  Video Quality Assessment (VQA) is a classic field in low-level visual perception. Recent studies in the image domain have demonstrated that Visual Question Answering (VQA) can enhance markedly low-level visual quality evaluation. We introduce the VQA2 Instruction dataset - the first visual question answering instruction dataset that focuses on video quality assessment. The VQA2 series models interleave visual and motion tokens to enhance the perception of spatial-temporal quality details in videos.
  arXiv  Detail & Related papers  (2024-11-06T09:39:52Z)
• LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models [53.64461404882853]
  Video quality assessment (VQA) algorithms are needed to monitor and optimize the quality of streaming videos. Here, we propose the first Large Multi-Modal Video Quality Assessment (LMM-VQA) model, which introduces a novel visual modeling strategy for quality-aware feature extraction.
  arXiv  Detail & Related papers  (2024-08-26T04:29:52Z)
• CLIPVQA:Video Quality Assessment via CLIP [56.94085651315878]
  We propose an efficient CLIP-based Transformer method for the VQA problem ( CLIPVQA) The proposed CLIPVQA achieves new state-of-the-art VQA performance and up to 37% better generalizability than existing benchmark VQA methods.
  arXiv  Detail & Related papers  (2024-07-06T02:32:28Z)
• Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video Quality Assessment [25.5501280406614]
  Video quality assessment (VQA) has attracted growing attention in recent years. The great expense of annotating large-scale VQA datasets has become the main obstacle for current deep-learning methods. An Adaptive Diverse Quality-aware feature Acquisition (Ada-DQA) framework is proposed to capture desired quality-related features.
  arXiv  Detail & Related papers  (2023-08-01T16:04:42Z)
• Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models [71.06007696593704]
  Blind quality assessment (BVQA) plays an indispensable role in monitoring and improving the end-users' viewing experience in real-world video-enabled media applications. As an experimental field, the improvements of BVQA models have been measured primarily on a few human-rated VQA datasets. We conduct a first-of-its-kind computational analysis of VQA datasets via minimalistic BVQA models.
  arXiv  Detail & Related papers  (2023-07-26T06:38:33Z)
• A Deep Learning based No-reference Quality Assessment Model for UGC Videos [44.00578772367465]
  Previous video quality assessment (VQA) studies either use the image recognition model or the image quality assessment (IQA) models to extract frame-level features of videos for quality regression. We propose a very simple but effective VQA model, which trains an end-to-end spatial feature extraction network to learn the quality-aware spatial feature representation from raw pixels of the video frames. With the better quality-aware features, we only use the simple multilayer perception layer (MLP) network to regress them into the chunk-level quality scores, and then the temporal average pooling strategy is adopted to obtain the video
  arXiv  Detail & Related papers  (2022-04-29T12:45:21Z)
• UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content [59.13821614689478]
  Blind quality prediction of in-the-wild videos is quite challenging, since the quality degradations of content are unpredictable, complicated, and often commingled. Here we contribute to advancing the problem by conducting a comprehensive evaluation of leading VQA models. By employing a feature selection strategy on top of leading VQA model features, we are able to extract 60 of the 763 statistical features used by the leading models. Our experimental results show that VIDEVAL achieves state-of-theart performance at considerably lower computational cost than other leading models.
  arXiv  Detail & Related papers  (2020-05-29T00:39:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.