Complexity Reduction of Learned In-Loop Filtering in Video Coding

• URL: http://arxiv.org/abs/2203.08650v2
• Date: Thu, 17 Mar 2022 10:09:17 GMT
• Title: Complexity Reduction of Learned In-Loop Filtering in Video Coding
• Authors: Woody Bayliss, Luka Murn, Ebroul Izquierdo, Qianni Zhang, Marta Mrak
• Abstract summary: In video coding, in-loop filters are applied on reconstructed video frames to enhance their perceptual quality, before storing the frames for output. The proposed method uses a novel combination of sparsity and structured pruning for complexity reduction of learned in-loop filters.
• Score: 12.06039429078762
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In video coding, in-loop filters are applied on reconstructed video frames to enhance their perceptual quality, before storing the frames for output. Conventional in-loop filters are obtained by hand-crafted methods. Recently, learned filters based on convolutional neural networks that utilize attention mechanisms have been shown to improve upon traditional techniques. However, these solutions are typically significantly more computationally expensive, limiting their potential for practical applications. The proposed method uses a novel combination of sparsity and structured pruning for complexity reduction of learned in-loop filters. This is done through a three-step training process of magnitude-guidedweight pruning, insignificant neuron identification and removal, and fine-tuning. Through initial tests we find that network parameters can be significantly reduced with a minimal impact on network performance.

Related papers

• In-Loop Filtering via Trained Look-Up Tables [45.6756570330982]
  In-loop filtering (ILF) is a key technology for removing the artifacts in image/video coding standards. We propose an efficient and practical in-loop filtering scheme by adopting the Look-up Table (LUT) Experimental results show that the ultrafast, very fast, and fast mode of the proposed method achieves on average 0.13%/0.34%/0.51%, and 0.10%/0.27%/0.39% BD-rate reduction.
  arXiv  Detail & Related papers  (2024-07-15T17:25:42Z)
• Neural Network Compression by Joint Sparsity Promotion and Redundancy Reduction [4.9613162734482215]
  This paper presents a novel training scheme based on composite constraints that prune redundant filters and minimize their effect on overall network learning via sparsity promotion. Our tests on several pixel-wise segmentation benchmarks show that the number of neurons and the memory footprint of networks in the test phase are significantly reduced without affecting performance.
  arXiv  Detail & Related papers  (2022-10-14T01:34:49Z)
• Understanding the Covariance Structure of Convolutional Filters [86.0964031294896]
  Recent ViT-inspired convolutional networks such as ConvMixer and ConvNeXt use large-kernel depthwise convolutions with notable structure. We first observe that such learned filters have highly-structured covariance matrices, and we find that covariances calculated from small networks may be used to effectively initialize a variety of larger networks.
  arXiv  Detail & Related papers  (2022-10-07T15:59:13Z)
• Asymptotic Soft Cluster Pruning for Deep Neural Networks [5.311178623385279]
  Filter pruning method introduces structural sparsity by removing selected filters. We propose a novel filter pruning method called Asymptotic Soft Cluster Pruning. Our method can achieve competitive results compared with many state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2022-06-16T13:58:58Z)
• Batch Normalization Tells You Which Filter is Important [49.903610684578716]
  We propose a simple yet effective filter pruning method by evaluating the importance of each filter based on the BN parameters of pre-trained CNNs. The experimental results on CIFAR-10 and ImageNet demonstrate that the proposed method can achieve outstanding performance.
  arXiv  Detail & Related papers  (2021-12-02T12:04:59Z)
• Learning Versatile Convolution Filters for Efficient Visual Recognition [125.34595948003745]
  This paper introduces versatile filters to construct efficient convolutional neural networks. We conduct theoretical analysis on network complexity and an efficient convolution scheme is introduced. Experimental results on benchmark datasets and neural networks demonstrate that our versatile filters are able to achieve comparable accuracy as that of original filters.
  arXiv  Detail & Related papers  (2021-09-20T06:07:14Z)
• Improved CNN-based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Coding [5.46121027847413]
  This paper introduces a novel explainable neural network-based inter-prediction scheme. A novel training framework enables each network branch to resemble a specific fractional shift. When implemented in the context of the Versatile Video Coding (VVC) test model, 0.77%, 1.27% and 2.25% BD-rate savings can be achieved.
  arXiv  Detail & Related papers  (2021-06-16T16:48:01Z)
• Unfolding Neural Networks for Compressive Multichannel Blind Deconvolution [71.29848468762789]
  We propose a learned-structured unfolding neural network for the problem of compressive sparse multichannel blind-deconvolution. In this problem, each channel's measurements are given as convolution of a common source signal and sparse filter. We demonstrate that our method is superior to classical structured compressive sparse multichannel blind-deconvolution methods in terms of accuracy and speed of sparse filter recovery.
  arXiv  Detail & Related papers  (2020-10-22T02:34:33Z)
• ALF: Autoencoder-based Low-rank Filter-sharing for Efficient Convolutional Neural Networks [63.91384986073851]
  We propose the autoencoder-based low-rank filter-sharing technique technique (ALF) ALF shows a reduction of 70% in network parameters, 61% in operations and 41% in execution time, with minimal loss in accuracy.
  arXiv  Detail & Related papers  (2020-07-27T09:01:22Z)
• Learning Sparse Filters in Deep Convolutional Neural Networks with a l1/l2 Pseudo-Norm [5.3791844634527495]
  Deep neural networks (DNNs) have proven to be efficient for numerous tasks, but come at a high memory and computation cost. Recent research has shown that their structure can be more compact without compromising their performance. We present a sparsity-inducing regularization term based on the ratio l1/l2 pseudo-norm defined on the filter coefficients.
  arXiv  Detail & Related papers  (2020-07-20T11:56:12Z)
• Computational optimization of convolutional neural networks using separated filters architecture [69.73393478582027]
  We consider a convolutional neural network transformation that reduces computation complexity and thus speedups neural network processing. Use of convolutional neural networks (CNN) is the standard approach to image recognition despite the fact they can be too computationally demanding.
  arXiv  Detail & Related papers  (2020-02-18T17:42:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.