CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement

• URL: http://arxiv.org/abs/2403.10362v2
• Date: Wed, 20 Nov 2024 02:58:25 GMT
• Title: CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement
• Authors: Qiang Zhu, Jinhua Hao, Yukang Ding, Yu Liu, Qiao Mo, Ming Sun, Chao Zhou, Shuyuan Zhu,
• Abstract summary: Coding Priors-Guided Aggregation (CPGA) network is developed to utilize temporal and spatial information from coding priors. To facilitate research in compressed video quality enhancement (VQE), we construct the Video Coding Priors dataset.
• Score: 11.862146973848558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, numerous approaches have achieved notable success in compressed video quality enhancement (VQE). However, these methods usually ignore the utilization of valuable coding priors inherently embedded in compressed videos, such as motion vectors and residual frames, which carry abundant temporal and spatial information. To remedy this problem, we propose the Coding Priors-Guided Aggregation (CPGA) network to utilize temporal and spatial information from coding priors. The CPGA mainly consists of an inter-frame temporal aggregation (ITA) module and a multi-scale non-local aggregation (MNA) module. Specifically, the ITA module aggregates temporal information from consecutive frames and coding priors, while the MNA module globally captures spatial information guided by residual frames. In addition, to facilitate research in VQE task, we newly construct the Video Coding Priors (VCP) dataset, comprising 300 videos with various coding priors extracted from corresponding bitstreams. It remedies the shortage of previous datasets on the lack of coding information. Experimental results demonstrate the superiority of our method compared to existing state-of-the-art methods. The code and dataset will be released at https://github.com/VQE-CPGA/CPGA.git .

Related papers

• Coding-Prior Guided Diffusion Network for Video Deblurring [47.77918791133459]
  We present a novel framework that effectively leverages both coding priors and generative diffusion priors for high-quality deblurring. Experiments demonstrate our method achieves state-of-the-art perceptual quality with up to 30% improvement in IQA metrics.
  arXiv  Detail & Related papers  (2025-04-16T16:14:43Z)
• Token-Efficient Long Video Understanding for Multimodal LLMs [101.70681093383365]
  STORM is a novel architecture incorporating a dedicated temporal encoder between the image encoder and the Video-LLMs. We show that STORM achieves state-of-the-art results across various long video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T06:17:38Z)
• High-Efficiency Neural Video Compression via Hierarchical Predictive Learning [27.41398149573729]
  Enhanced Deep Hierarchical Video Compression-DHVC 2.0- introduces superior compression performance and impressive complexity efficiency. Uses hierarchical predictive coding to transform each video frame into multiscale representations. Supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss.
  arXiv  Detail & Related papers  (2024-10-03T15:40:58Z)
• Boosting Neural Representations for Videos with a Conditional Decoder [28.073607937396552]
  Implicit neural representations (INRs) have emerged as a promising approach for video storage and processing. This paper introduces a universal boosting framework for current implicit video representation approaches.
  arXiv  Detail & Related papers  (2024-02-28T08:32:19Z)
• Local Compressed Video Stream Learning for Generic Event Boundary Detection [25.37983456118522]
  Event boundary detection aims to localize the generic, taxonomy-free event boundaries that segment videos into chunks. Existing methods typically require video frames to be decoded before feeding into the network. We propose a novel event boundary detection method that is fully end-to-end leveraging rich information in the compressed domain.
  arXiv  Detail & Related papers  (2023-09-27T06:49:40Z)
• Co-attention Propagation Network for Zero-Shot Video Object Segmentation [91.71692262860323]
  Zero-shot object segmentation (ZS-VOS) aims to segment objects in a video sequence without prior knowledge of these objects. Existing ZS-VOS methods often struggle to distinguish between foreground and background or to keep track of the foreground in complex scenarios. We propose an encoder-decoder-based hierarchical co-attention propagation network (HCPN) capable of tracking and segmenting objects.
  arXiv  Detail & Related papers  (2023-04-08T04:45:48Z)
• You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos [56.676761067861236]
  Given an untrimmed video, temporal sentence grounding aims to locate a target moment semantically according to a sentence query. Previous respectable works have made decent success, but they only focus on high-level visual features extracted from decoded frames. We propose a new setting, compressed-domain TSG, which directly utilizes compressed videos rather than fully-decompressed frames as the visual input.
  arXiv  Detail & Related papers  (2023-03-14T12:53:27Z)
• Exploring Long- and Short-Range Temporal Information for Learned Video Compression [54.91301930491466]
  We focus on exploiting the unique characteristics of video content and exploring temporal information to enhance compression performance. For long-range temporal information exploitation, we propose temporal prior that can update continuously within the group of pictures (GOP) during inference. In that case temporal prior contains valuable temporal information of all decoded images within the current GOP. In detail, we design a hierarchical structure to achieve multi-scale compensation.
  arXiv  Detail & Related papers  (2022-08-07T15:57:18Z)
• STIP: A SpatioTemporal Information-Preserving and Perception-Augmented Model for High-Resolution Video Prediction [78.129039340528]
  We propose a Stemporal Information-Preserving and Perception-Augmented Model (STIP) to solve the above two problems. The proposed model aims to preserve thetemporal information for videos during the feature extraction and the state transitions. Experimental results show that the proposed STIP can predict videos with more satisfactory visual quality compared with a variety of state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-09T09:49:04Z)
• End-to-End Compressed Video Representation Learning for Generic Event Boundary Detection [31.31508043234419]
  We propose a new end-to-end compressed video representation learning for event boundary detection. We first use the ConvNets to extract features of the I-frames in the GOPs. After that, a light-weight spatial-channel compressed encoder is designed to compute the feature representations of the P-frames. A temporal contrastive module is proposed to determine the event boundaries of video sequences.
  arXiv  Detail & Related papers  (2022-03-29T08:27:48Z)
• A Coding Framework and Benchmark towards Low-Bitrate Video Understanding [63.05385140193666]
  We propose a traditional-neural mixed coding framework that takes advantage of both traditional codecs and neural networks (NNs) The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved. We build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach.
  arXiv  Detail & Related papers  (2022-02-06T16:29:15Z)
• Transcoded Video Restoration by Temporal Spatial Auxiliary Network [64.63157339057912]
  We propose a new method, temporal spatial auxiliary network (TSAN), for transcoded video restoration. The experimental results demonstrate that the performance of the proposed method is superior to that of the previous techniques.
  arXiv  Detail & Related papers  (2021-12-15T08:10:23Z)
• End-to-End Learning for Video Frame Compression with Self-Attention [25.23586503813838]
  We propose an end-to-end learned system for compressing video frames. Our system learns deep embeddings of frames and encodes their difference in latent space. In our experiments, we show that the proposed system achieves high compression rates and high objective visual quality.
  arXiv  Detail & Related papers  (2020-04-20T12:11:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.