Multi-Modality Deep Network for JPEG Artifacts Reduction

• URL: http://arxiv.org/abs/2305.02760v1
• Date: Thu, 4 May 2023 11:54:02 GMT
• Title: Multi-Modality Deep Network for JPEG Artifacts Reduction
• Authors: Xuhao Jiang, Weimin Tan, Qing Lin, Chenxi Ma, Bo Yan, Liquan Shen
• Abstract summary: We propose a multimodal fusion learning method for text-guided JPEG artifacts reduction. Our method can obtain better deblocking results compared to the state-of-the-art methods.
• Score: 33.02405073842042
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, many convolutional neural network-based models are designed for JPEG artifacts reduction, and have achieved notable progress. However, few methods are suitable for extreme low-bitrate image compression artifacts reduction. The main challenge is that the highly compressed image loses too much information, resulting in reconstructing high-quality image difficultly. To address this issue, we propose a multimodal fusion learning method for text-guided JPEG artifacts reduction, in which the corresponding text description not only provides the potential prior information of the highly compressed image, but also serves as supplementary information to assist in image deblocking. We fuse image features and text semantic features from the global and local perspectives respectively, and design a contrastive loss built upon contrastive learning to produce visually pleasing results. Extensive experiments, including a user study, prove that our method can obtain better deblocking results compared to the state-of-the-art methods.

Related papers

• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• ENTED: Enhanced Neural Texture Extraction and Distribution for Reference-based Blind Face Restoration [51.205673783866146]
  We present ENTED, a new framework for blind face restoration that aims to restore high-quality and realistic portrait images. We utilize a texture extraction and distribution framework to transfer high-quality texture features between the degraded input and reference image. The StyleGAN-like architecture in our framework requires high-quality latent codes to generate realistic images.
  arXiv  Detail & Related papers  (2024-01-13T04:54:59Z)
• Transferable Learned Image Compression-Resistant Adversarial Perturbations [66.46470251521947]
  Adversarial attacks can readily disrupt the image classification system, revealing the vulnerability of DNN-based recognition tasks. We introduce a new pipeline that targets image classification models that utilize learned image compressors as pre-processing modules.
  arXiv  Detail & Related papers  (2024-01-06T03:03:28Z)
• Multi-Modality Deep Network for Extreme Learned Image Compression [31.532613540054697]
  We propose a multimodal machine learning method for text-guided image compression, in which semantic information of text is used as prior information to guide image compression performance. In addition, we adopt the image-text attention module and image-request complement module to better fuse image and text features, and propose an improved multimodal semantic-consistent loss to produce semantically complete reconstructions.
  arXiv  Detail & Related papers  (2023-04-26T14:22:59Z)
• High-Perceptual Quality JPEG Decoding via Posterior Sampling [13.238373528922194]
  We propose a different paradigm for JPEG artifact correction. We aim to obtain sharp, detailed and visually reconstructed images, while being consistent with the compressed input. Our solution offers a diverse set of plausible and fast reconstructions for a given input with perfect consistency.
  arXiv  Detail & Related papers  (2022-11-21T19:47:59Z)
• Extreme Generative Image Compression by Learning Text Embedding from Diffusion Models [13.894251782142584]
  We propose a generative image compression method that demonstrates the potential of saving an image as a short text embedding. Our method outperforms other state-of-the-art deep learning methods in terms of both perceptual quality and diversity.
  arXiv  Detail & Related papers  (2022-11-14T22:54:19Z)
• Convolutional Neural Network (CNN) to reduce construction loss in JPEG compression caused by Discrete Fourier Transform (DFT) [0.0]
  Convolutional Neural Networks (CNN) have received more attention than most other types of deep neural networks. In this work, an effective image compression method is purposed using autoencoders.
  arXiv  Detail & Related papers  (2022-08-26T12:46:16Z)
• Learning a Single Model with a Wide Range of Quality Factors for JPEG Image Artifacts Removal [24.25688335628976]
  Lossy compression brings artifacts into the compressed image and degrades the visual quality. In this paper, we propose a highly robust compression artifacts removal network. Our proposed network is a single model approach that can be trained for handling a wide range of quality factors.
  arXiv  Detail & Related papers  (2020-09-15T08:16:58Z)
• Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images [54.40852143927333]
  Lossy image compression is pervasively conducted to save communication bandwidth, resulting in undesirable compression artifacts. We propose a resource-efficient blind quality enhancement (RBQE) approach for compressed images. Our approach can automatically decide to terminate or continue enhancement according to the assessed quality of enhanced images.
  arXiv  Detail & Related papers  (2020-06-30T07:38:47Z)
• Quantization Guided JPEG Artifact Correction [69.04777875711646]
  We develop a novel architecture for artifact correction using the JPEG files quantization matrix. This allows our single model to achieve state-of-the-art performance over models trained for specific quality settings.
  arXiv  Detail & Related papers  (2020-04-17T00:10:08Z)
• Learning End-to-End Lossy Image Compression: A Benchmark [90.35363142246806]
  We first conduct a comprehensive literature survey of learned image compression methods. We describe milestones in cutting-edge learned image-compression methods, review a broad range of existing works, and provide insights into their historical development routes. By introducing a coarse-to-fine hyperprior model for entropy estimation and signal reconstruction, we achieve improved rate-distortion performance.
  arXiv  Detail & Related papers  (2020-02-10T13:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.