Neural Video Compression with In-Loop Contextual Filtering and Out-of-Loop Reconstruction Enhancement

• URL: http://arxiv.org/abs/2509.04051v1
• Date: Thu, 04 Sep 2025 09:29:30 GMT
• Title: Neural Video Compression with In-Loop Contextual Filtering and Out-of-Loop Reconstruction Enhancement
• Authors: Yaojun Wu, Chaoyi Lin, Yiming Wang, Semih Esenlik, Zhaobin Zhang, Kai Zhang, Li Zhang,
• Abstract summary: This paper explores the application of enhancement filtering techniques in neural video compression.<n>In-loop contextual filtering refines the temporal context by mitigating error propagation during frame-by-frame encoding.<n>Out-of-loop reconstruction enhancement is employed to refine the quality of reconstructed frames, providing a simple yet effective improvement in coding efficiency.
• Score: 11.849261047358072
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper explores the application of enhancement filtering techniques in neural video compression. Specifically, we categorize these techniques into in-loop contextual filtering and out-of-loop reconstruction enhancement based on whether the enhanced representation affects the subsequent coding loop. In-loop contextual filtering refines the temporal context by mitigating error propagation during frame-by-frame encoding. However, its influence on both the current and subsequent frames poses challenges in adaptively applying filtering throughout the sequence. To address this, we introduce an adaptive coding decision strategy that dynamically determines filtering application during encoding. Additionally, out-of-loop reconstruction enhancement is employed to refine the quality of reconstructed frames, providing a simple yet effective improvement in coding efficiency. To the best of our knowledge, this work presents the first systematic study of enhancement filtering in the context of conditional-based neural video compression. Extensive experiments demonstrate a 7.71% reduction in bit rate compared to state-of-the-art neural video codecs, validating the effectiveness of the proposed approach.

Related papers

• Content Adaptive based Motion Alignment Framework for Learned Video Compression [72.13599533975413]
  This paper proposes a content adaptive based motion alignment framework.<n>We first introduce a two-stage flow-guided deformable warping mechanism that refines motion compensation with coarse-to-fine offset prediction and mask modulation.<n>Second, we propose a multi-reference quality aware strategy that adjusts distortion weights based on reference quality, and applies it to hierarchical training to reduce error propagation.<n>Third, we integrate a training-free module that downsamples frames by motion magnitude and resolution to obtain smooth motion estimation.
  arXiv  Detail & Related papers  (2025-12-15T02:51:47Z)
• Prediction and Reference Quality Adaptation for Learned Video Compression [54.58691829087094]
  Temporal prediction is one of the most important technologies for video compression.<n>Traditional video codecs adaptively decide the optimal coding mode according to the prediction quality and reference quality.<n>We propose a confidence-based prediction quality adaptation (PQA) module and a reference quality adaptation (RQA) module.
  arXiv  Detail & Related papers  (2024-06-20T09:03:26Z)
• Competitive Learning for Achieving Content-specific Filters in Video Coding for Machines [5.155405463139862]
  This paper investigates the efficacy of jointly optimizing content-specific post-processing filters to adapt a human oriented video/image into a machine vision task. We propose a novel training strategy based on competitive learning principles. Experiments on the OpenImages dataset show an improvement in the BD-rate reduction from -41.3% to -44.6%.
  arXiv  Detail & Related papers  (2024-06-18T07:45:57Z)
• Compression-Realized Deep Structural Network for Video Quality Enhancement [78.13020206633524]
  This paper focuses on the task of quality enhancement for compressed videos. Most of the existing methods lack a structured design to optimally leverage the priors within compression codecs. A new paradigm is urgently needed for a more conscious'' process of quality enhancement.
  arXiv  Detail & Related papers  (2024-05-10T09:18:17Z)
• Boosting Neural Representations for Videos with a Conditional Decoder [28.073607937396552]
  Implicit neural representations (INRs) have emerged as a promising approach for video storage and processing. This paper introduces a universal boosting framework for current implicit video representation approaches.
  arXiv  Detail & Related papers  (2024-02-28T08:32:19Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)
• Complexity Reduction of Learned In-Loop Filtering in Video Coding [12.06039429078762]
  In video coding, in-loop filters are applied on reconstructed video frames to enhance their perceptual quality, before storing the frames for output. The proposed method uses a novel combination of sparsity and structured pruning for complexity reduction of learned in-loop filters.
  arXiv  Detail & Related papers  (2022-03-16T14:34:41Z)
• End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation [33.54844063875569]
  We propose a hybrid motion compensation (HMC) method that adaptively combines the predictions generated by two approaches. Specifically, we generate a compoundtemporal representation (STR) through a recurrent information aggregation (RIA) module. We further design a one-to-many decoder pipeline to generate multiple predictions from the CSTR, including vector-based resampling, adaptive kernel-based resampling, compensation mode selection maps and texture enhancements.
  arXiv  Detail & Related papers  (2021-08-05T19:43:32Z)
• A Global Appearance and Local Coding Distortion based Fusion Framework for CNN based Filtering in Video Coding [15.778380865885842]
  In-loop filtering is used in video coding to process the reconstructed frame in order to remove blocking artifacts. In this paper, we address the filtering problem from two aspects, global appearance restoration for disrupted texture and local coding distortion restoration caused by fixed pipeline of coding. A three-stream global appearance and local coding distortion based fusion network is developed with a high-level global feature stream, a high-level local feature stream and a low-level local feature stream.
  arXiv  Detail & Related papers  (2021-06-24T03:08:44Z)
• Unfolding Neural Networks for Compressive Multichannel Blind Deconvolution [71.29848468762789]
  We propose a learned-structured unfolding neural network for the problem of compressive sparse multichannel blind-deconvolution. In this problem, each channel's measurements are given as convolution of a common source signal and sparse filter. We demonstrate that our method is superior to classical structured compressive sparse multichannel blind-deconvolution methods in terms of accuracy and speed of sparse filter recovery.
  arXiv  Detail & Related papers  (2020-10-22T02:34:33Z)
• Efficient Adaptation of Neural Network Filter for Video Compression [10.769305738505071]
  We present an efficient finetuning methodology for neural-network filters. The fine-tuning is performed at encoder side to adapt the neural network to the specific content that is being encoded. The proposed method achieves much faster than conventional finetuning approaches.
  arXiv  Detail & Related papers  (2020-07-28T14:24:28Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.