VCISR: Blind Single Image Super-Resolution with Video Compression Synthetic Data

• URL: http://arxiv.org/abs/2311.00996v2
• Date: Thu, 23 Nov 2023 03:58:04 GMT
• Title: VCISR: Blind Single Image Super-Resolution with Video Compression Synthetic Data
• Authors: Boyang Wang, Bowen Liu, Shiyu Liu, Fengyu Yang
• Abstract summary: We present a video compression-based degradation model to synthesize low-resolution image data in the blind SISR task. Our proposed image synthesizing method is widely applicable to existing image datasets. By introducing video coding artifacts to SISR degradation models, neural networks can super-resolve images with the ability to restore video compression degradations.
• Score: 18.877077302923713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the blind single image super-resolution (SISR) task, existing works have been successful in restoring image-level unknown degradations. However, when a single video frame becomes the input, these works usually fail to address degradations caused by video compression, such as mosquito noise, ringing, blockiness, and staircase noise. In this work, we for the first time, present a video compression-based degradation model to synthesize low-resolution image data in the blind SISR task. Our proposed image synthesizing method is widely applicable to existing image datasets, so that a single degraded image can contain distortions caused by the lossy video compression algorithms. This overcomes the leak of feature diversity in video data and thus retains the training efficiency. By introducing video coding artifacts to SISR degradation models, neural networks can super-resolve images with the ability to restore video compression degradations, and achieve better results on restoring generic distortions caused by image compression as well. Our proposed approach achieves superior performance in SOTA no-reference Image Quality Assessment, and shows better visual quality on various datasets. In addition, we evaluate the SISR neural network trained with our degradation model on video super-resolution (VSR) datasets. Compared to architectures specifically designed for the VSR purpose, our method exhibits similar or better performance, evidencing that the presented strategy on infusing video-based degradation is generalizable to address more complicated compression artifacts even without temporal cues.

Related papers

• Implicit Neural Representation for Videos Based on Residual Connection [0.0]
  We propose a method that uses low-resolution frames as residual connection that is considered effective for image reconstruction. Experimental results show that our method outperforms the existing method, HNeRV, in PSNR for 46 of the 49 videos.
  arXiv  Detail & Related papers  (2024-06-15T10:10:48Z)
• Improving Multi-generation Robustness of Learned Image Compression [16.86614420872084]
  We show that LIC can achieve comparable performance to the first compression of BPG even after 50 times reencoding without any change of the network structure.
  arXiv  Detail & Related papers  (2022-10-31T03:26:11Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)
• Reducing Redundancy in the Bottleneck Representation of the Autoencoders [98.78384185493624]
  Autoencoders are a type of unsupervised neural networks, which can be used to solve various tasks. We propose a scheme to explicitly penalize feature redundancies in the bottleneck representation. We tested our approach across different tasks: dimensionality reduction using three different dataset, image compression using the MNIST dataset, and image denoising using fashion MNIST.
  arXiv  Detail & Related papers  (2022-02-09T18:48:02Z)
• Super-Resolving Compressed Video in Coding Chain [27.994055823226848]
  We present a mixed-resolution coding framework, which cooperates with a reference-based DCNN. In this novel coding chain, the reference-based DCNN learns the direct mapping from low-resolution (LR) compressed video to their high-resolution (HR) clean version at the decoder side.
  arXiv  Detail & Related papers  (2021-03-26T03:39:54Z)
• Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images [54.40852143927333]
  Lossy image compression is pervasively conducted to save communication bandwidth, resulting in undesirable compression artifacts. We propose a resource-efficient blind quality enhancement (RBQE) approach for compressed images. Our approach can automatically decide to terminate or continue enhancement according to the assessed quality of enhanced images.
  arXiv  Detail & Related papers  (2020-06-30T07:38:47Z)
• Modeling Lost Information in Lossy Image Compression [72.69327382643549]
  Lossy image compression is one of the most commonly used operators for digital images. We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
  arXiv  Detail & Related papers  (2020-06-22T04:04:56Z)
• Attention Based Real Image Restoration [48.933507352496726]
  Deep convolutional neural networks perform better on images containing synthetic degradations. This paper proposes a novel single-stage blind real image restoration network (R$2$Net)
  arXiv  Detail & Related papers  (2020-04-26T04:21:49Z)
• Learning Better Lossless Compression Using Lossy Compression [100.50156325096611]
  We leverage the powerful lossy image compression algorithm BPG to build a lossless image compression system. We model the distribution of the residual with a convolutional neural network-based probabilistic model that is conditioned on the BPG reconstruction. Finally, the image is stored using the concatenation of the bitstreams produced by BPG and the learned residual coder.
  arXiv  Detail & Related papers  (2020-03-23T11:21:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.