M-LVC: Multiple Frames Prediction for Learned Video Compression
- URL: http://arxiv.org/abs/2004.10290v1
- Date: Tue, 21 Apr 2020 20:42:02 GMT
- Title: M-LVC: Multiple Frames Prediction for Learned Video Compression
- Authors: Jianping Lin, Dong Liu, Houqiang Li, Feng Wu
- Abstract summary: We propose an end-to-end learned video compression scheme for low-latency scenarios.
In our scheme, the motion vector (MV) field is calculated between the current frame and the previous one.
Experimental results show that the proposed method outperforms the existing learned video compression methods for low-latency mode.
- Score: 111.50760486258993
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We propose an end-to-end learned video compression scheme for low-latency
scenarios. Previous methods are limited in using the previous one frame as
reference. Our method introduces the usage of the previous multiple frames as
references. In our scheme, the motion vector (MV) field is calculated between
the current frame and the previous one. With multiple reference frames and
associated multiple MV fields, our designed network can generate more accurate
prediction of the current frame, yielding less residual. Multiple reference
frames also help generate MV prediction, which reduces the coding cost of MV
field. We use two deep auto-encoders to compress the residual and the MV,
respectively. To compensate for the compression error of the auto-encoders, we
further design a MV refinement network and a residual refinement network,
taking use of the multiple reference frames as well. All the modules in our
scheme are jointly optimized through a single rate-distortion loss function. We
use a step-by-step training strategy to optimize the entire scheme.
Experimental results show that the proposed method outperforms the existing
learned video compression methods for low-latency mode. Our method also
performs better than H.265 in both PSNR and MS-SSIM. Our code and models are
publicly available.
Related papers
- IBVC: Interpolation-driven B-frame Video Compression [68.18440522300536]
B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction.
Previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation.
We propose a simple yet effective structure called Interpolation-B-frame Video Compression (IBVC) to address these issues.
arXiv Detail & Related papers (2023-09-25T02:45:51Z) - Predictive Coding For Animation-Based Video Compression [13.161311799049978]
We propose a predictive coding scheme which uses image animation as a predictor, and codes the residual with respect to the actual target frame.
Our experiments indicate a significant gain, in excess of 70% compared to the HEVC video standard and over 30% compared to VVC.
arXiv Detail & Related papers (2023-07-09T14:40:54Z) - Scene Matters: Model-based Deep Video Compression [13.329074811293292]
We propose a model-based video compression (MVC) framework that regards scenes as the fundamental units for video sequences.
Our proposed MVC directly models novel intensity variation of the entire video sequence in one scene, seeking non-redundant representations instead of reducing redundancy.
Our method achieves up to a 20% reduction compared to the latest video standard H.266 and is more efficient in decoding than existing video coding strategies.
arXiv Detail & Related papers (2023-03-08T13:15:19Z) - Butterfly: Multiple Reference Frames Feature Propagation Mechanism for
Neural Video Compression [17.073251238499314]
We present a more reasonable multi-reference frames propagation mechanism for neural video compression.
Our method can significantly outperform the previous state-of-the-art (SOTA)
Our neural dataset can achieve -7.6% save on HEVC Class D when compared with our base single-reference frame model.
arXiv Detail & Related papers (2023-03-06T08:19:15Z) - Advancing Learned Video Compression with In-loop Frame Prediction [177.67218448278143]
In this paper, we propose an Advanced Learned Video Compression (ALVC) approach with the in-loop frame prediction module.
The predicted frame can serve as a better reference than the previously compressed frame, and therefore it benefits the compression performance.
The experiments show the state-of-the-art performance of our ALVC approach in learned video compression.
arXiv Detail & Related papers (2022-11-13T19:53:14Z) - Learning Trajectory-Aware Transformer for Video Super-Resolution [50.49396123016185]
Video super-resolution aims to restore a sequence of high-resolution (HR) frames from their low-resolution (LR) counterparts.
Existing approaches usually align and aggregate video frames from limited adjacent frames.
We propose a novel Transformer for Video Super-Resolution (TTVSR)
arXiv Detail & Related papers (2022-04-08T03:37:39Z) - End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional
Video Compression [10.885590093103344]
Learned VC allows end-to-end rate-distortion (R-D) optimized training of nonlinear transform, motion and entropy model simultaneously.
This paper proposes a learned hierarchical bi-directional video (LHBDC) that combines the benefits of hierarchical motion-sampling and end-to-end optimization.
arXiv Detail & Related papers (2021-12-17T14:30:22Z) - Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames.
We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs.
We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
arXiv Detail & Related papers (2020-08-20T20:01:59Z) - Learning for Video Compression with Recurrent Auto-Encoder and Recurrent
Probability Model [164.7489982837475]
This paper proposes a Recurrent Learned Video Compression (RLVC) approach with the Recurrent Auto-Encoder (RAE) and Recurrent Probability Model ( RPM)
The RAE employs recurrent cells in both the encoder and decoder to exploit the temporal correlation among video frames.
Our approach achieves the state-of-the-art learned video compression performance in terms of both PSNR and MS-SSIM.
arXiv Detail & Related papers (2020-06-24T08:46:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.