Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression

• URL: http://arxiv.org/abs/2401.14007v2
• Date: Fri, 25 Oct 2024 07:08:45 GMT
• Title: Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression
• Authors: Daxin Li, Yuanchao Bai, Kai Wang, Junjun Jiang, Xianming Liu,
• Abstract summary: This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
• Score: 58.618625678054826
• License:
• Abstract: Recent advancements in neural compression have surpassed traditional codecs in PSNR and MS-SSIM measurements. However, at low bit-rates, these methods can introduce visually displeasing artifacts, such as blurring, color shifting, and texture loss, thereby compromising perceptual quality of images. To address these issues, this study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss, to enhance the perceptual quality of image reconstructions. Additionally, we have implemented a latent refinement process to generate content-aware latent codes. These codes adhere to bit-rate constraints, balance the trade-off between distortion and fidelity, and prioritize bit allocation to regions of greater importance. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression. On CLIC2024 validation set, our approach achieves a 62% bitrate saving compared to MS-ILLM under FID metric.

Related papers

• Enhancing Perception Quality in Remote Sensing Image Compression via Invertible Neural Network [10.427300958330816]
  Decoding remote sensing images to achieve high perceptual quality, particularly at lows, remains a significant challenge. We propose the invertible neural network-based remote sensing image compression (INN-RSIC) method. Our INN-RSIC significantly outperforms the existing state-of-the-art traditional and deep learning-based image compression methods in terms of perception quality.
  arXiv  Detail & Related papers  (2024-05-17T03:52:37Z)
• Neural Image Compression with Quantization Rectifier [7.097091519502871]
  We develop a novel quantization (QR) method for image compression that leverages image feature correlation to mitigate the impact of quantization. Our method designs a neural network architecture that predicts unquantized features from the quantized ones. In evaluation, we integrate QR into state-of-the-art neural image codecs and compare enhanced models and baselines on the widely-used Kodak benchmark.
  arXiv  Detail & Related papers  (2024-03-25T22:26:09Z)
• Extreme Image Compression using Fine-tuned VQGANs [43.43014096929809]
  We introduce vector quantization (VQ)-based generative models into the image compression domain. The codebook learned by the VQGAN model yields a strong expressive capacity. The proposed framework outperforms state-of-the-art codecs in terms of perceptual quality-oriented metrics.
  arXiv  Detail & Related papers  (2023-07-17T06:14:19Z)
• You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
  Learned image compression (LIC) methods have experienced significant progress during recent years. LIC methods fail to explicitly explore the image structure and texture components crucial for image compression. We present DA-Mask that samples visible patches based on the structure and texture of original images. We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
  arXiv  Detail & Related papers  (2023-06-27T15:36:22Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)
• Perceptually Optimizing Deep Image Compression [53.705543593594285]
  Mean squared error (MSE) and $ell_p$ norms have largely dominated the measurement of loss in neural networks. We propose a different proxy approach to optimize image analysis networks against quantitative perceptual models.
  arXiv  Detail & Related papers  (2020-07-03T14:33:28Z)
• Attention Based Real Image Restoration [48.933507352496726]
  Deep convolutional neural networks perform better on images containing synthetic degradations. This paper proposes a novel single-stage blind real image restoration network (R$2$Net)
  arXiv  Detail & Related papers  (2020-04-26T04:21:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.