MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment

• URL: http://arxiv.org/abs/2504.16003v1
• Date: Tue, 22 Apr 2025 16:08:23 GMT
• Title: MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment
• Authors: Yachun Mi, Yu Li, Weicheng Meng, Chaofeng Chen, Chen Hui, Shaohui Liu,
• Abstract summary: We present MVQA, a Mamba-based model designed for efficient video quality assessment (VQA)<n>USDS combines semantic patch sampling from low-resolution videos and distortion patch sampling from original-resolution videos.<n>Experiments show that the proposed MVQA, equipped with USDS, achieve comparable performance to state-of-the-art methods.
• Score: 24.053542031123985
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid growth of long-duration, high-definition videos has made efficient video quality assessment (VQA) a critical challenge. Existing research typically tackles this problem through two main strategies: reducing model parameters and resampling inputs. However, light-weight Convolution Neural Networks (CNN) and Transformers often struggle to balance efficiency with high performance due to the requirement of long-range modeling capabilities. Recently, the state-space model, particularly Mamba, has emerged as a promising alternative, offering linear complexity with respect to sequence length. Meanwhile, efficient VQA heavily depends on resampling long sequences to minimize computational costs, yet current resampling methods are often weak in preserving essential semantic information. In this work, we present MVQA, a Mamba-based model designed for efficient VQA along with a novel Unified Semantic and Distortion Sampling (USDS) approach. USDS combines semantic patch sampling from low-resolution videos and distortion patch sampling from original-resolution videos. The former captures semantically dense regions, while the latter retains critical distortion details. To prevent computation increase from dual inputs, we propose a fusion mechanism using pre-defined masks, enabling a unified sampling strategy that captures both semantic and quality information without additional computational burden. Experiments show that the proposed MVQA, equipped with USDS, achieve comparable performance to state-of-the-art methods while being $2\times$ as fast and requiring only $1/5$ GPU memory.

Related papers

• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations.<n> Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations.<n> Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• VADMamba: Exploring State Space Models for Fast Video Anomaly Detection [4.874215132369157]
  VQ-Mamba Unet (VQ-MaU) framework incorporates a Vector Quantization (VQ) layer and Mamba-based Non-negative Visual State Space (NVSS) block.<n>Results validate the efficacy of the proposed VADMamba across three benchmark datasets.
  arXiv  Detail & Related papers  (2025-03-27T05:38:12Z)
• Training-free Diffusion Acceleration with Bottleneck Sampling [37.9135035506567]
  Bottleneck Sampling is a training-free framework that leverages low-resolution priors to reduce computational overhead while preserving output fidelity.<n>It accelerates inference by up to 3$times$ for image generation and 2.5$times$ for video generation, all while maintaining output quality comparable to the standard full-resolution sampling process.
  arXiv  Detail & Related papers  (2025-03-24T17:59:02Z)
• Low-Resource Video Super-Resolution using Memory, Wavelets, and Deformable Convolutions [3.018928786249079]
  Video super-resolution (VSR) remains a formidable challenge in its adoption for deployment on resource-constrained edge devices.<n>We propose a novel lightweight and parameter-efficient neural architecture for VSR that achieves state-of-the-art reconstruction accuracy with just 2.3 million parameters.
  arXiv  Detail & Related papers  (2025-02-03T20:46:15Z)
• MS-Temba : Multi-Scale Temporal Mamba for Efficient Temporal Action Detection [11.534493974662304]
  Temporal Action Detection (TAD) in untrimmed videos requires models that can efficiently process long-duration videos. We propose Multi-Scale Temporal Mamba (MS-Temba), the first Mamba-based architecture specifically designed for densely labeled TAD tasks. MS-Temba achieves state-of-the-art performance on long-duration videos, remains competitive on shorter segments, and reduces model complexity by 88%.
  arXiv  Detail & Related papers  (2025-01-10T17:52:47Z)
• Look Every Frame All at Once: Video-Ma$^2$mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing [52.050036778325094]
  Video-Ma$2$mba is a novel architecture that incorporates State Space Models (SSMs) within the Mamba-2 framework.<n>Our approach significantly reduces the memory footprint compared to standard gradient checkpointing.<n>By maintaining a detailed capture of temporal dynamics, our model improves the accuracy and relevance of responses in long video understanding tasks.
  arXiv  Detail & Related papers  (2024-11-29T04:12:13Z)
• PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution [95.98801201266099]
  Diffusion-based image super-resolution (SR) models have shown superior performance at the cost of multiple denoising steps.<n>We propose a novel post-training quantization approach with adaptive scale in one-step diffusion (OSD) image SR, PassionSR.<n>Our PassionSR achieves significant advantages over recent leading low-bit quantization methods for image SR.
  arXiv  Detail & Related papers  (2024-11-26T04:49:42Z)
• Multimodal Instruction Tuning with Hybrid State Space Models [25.921044010033267]
  Long context is crucial for enhancing the recognition and understanding capabilities of multimodal large language models. We propose a novel approach using a hybrid transformer-MAMBA model to efficiently handle long contexts in multimodal applications. Our model enhances inference efficiency for high-resolution images and high-frame-rate videos by about 4 times compared to current models.
  arXiv  Detail & Related papers  (2024-11-13T18:19:51Z)
• Cross-Scan Mamba with Masked Training for Robust Spectral Imaging [51.557804095896174]
  We propose the Cross-Scanning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding.<n>Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.
  arXiv  Detail & Related papers  (2024-08-01T15:14:10Z)
• Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment [60.57703721744873]
  The increased resolution of real-world videos presents a dilemma between efficiency and accuracy for deep Video Quality Assessment (VQA) In this work, we propose a unified scheme, spatial-temporal grid mini-cube sampling (St-GMS) to get a novel type of sample, named fragments. With fragments and FANet, the proposed efficient end-to-end FAST-VQA and FasterVQA achieve significantly better performance than existing approaches on all VQA benchmarks.
  arXiv  Detail & Related papers  (2022-10-11T11:38:07Z)
• FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling [54.31355080688127]
  Current deep video quality assessment (VQA) methods are usually with high computational costs when evaluating high-resolution videos. We propose Grid Mini-patch Sampling (GMS), which allows consideration of local quality by sampling patches at their raw resolution. We build the Fragment Attention Network (FANet) specially designed to accommodate fragments as inputs. FAST-VQA improves state-of-the-art accuracy by around 10% while reducing 99.5% FLOPs on 1080P high-resolution videos.
  arXiv  Detail & Related papers  (2022-07-06T11:11:43Z)
• Pruning Self-attentions into Convolutional Layers in Single Path [89.55361659622305]
  Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. We propose Single-Path Vision Transformer pruning (SPViT) to efficiently and automatically compress the pre-trained ViTs. Our SPViT can trim 52.0% FLOPs for DeiT-B and get an impressive 0.6% top-1 accuracy gain simultaneously.
  arXiv  Detail & Related papers  (2021-11-23T11:35:54Z)
• Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTM [0.0]
  We propose an efficient two-stream deep learning architecture leveraging Separable Convolutional LSTM (SepConvLSTM) and pre-trained MobileNet. SepConvLSTM is constructed by replacing convolution operation at each gate of ConvLSTM with a depthwise separable convolution. Our model outperforms the accuracy on the larger and more challenging RWF-2000 dataset by more than a 2% margin.
  arXiv  Detail & Related papers  (2021-02-21T12:01:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.