Perceptual Learned Image Compression via End-to-End JND-Based Optimization

• URL: http://arxiv.org/abs/2402.02836v1
• Date: Mon, 5 Feb 2024 09:45:38 GMT
• Title: Perceptual Learned Image Compression via End-to-End JND-Based Optimization
• Authors: Farhad Pakdaman, Sanaz Nami, and Moncef Gabbouj
• Abstract summary: Emerging Learned image Compression (LC) achieves significant improvements in coding efficiency by end-to-end training of neural networks for compression. Perceptual optimization of LC to comply with the Human Visual System (HVS) is among such criteria, which has not been fully explored yet. This paper proposes a novel framework to integrate Just Noticeable Distortion (JND) principles into LC.
• Score: 15.173265255635219
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Emerging Learned image Compression (LC) achieves significant improvements in coding efficiency by end-to-end training of neural networks for compression. An important benefit of this approach over traditional codecs is that any optimization criteria can be directly applied to the encoder-decoder networks during training. Perceptual optimization of LC to comply with the Human Visual System (HVS) is among such criteria, which has not been fully explored yet. This paper addresses this gap by proposing a novel framework to integrate Just Noticeable Distortion (JND) principles into LC. Leveraging existing JND datasets, three perceptual optimization methods are proposed to integrate JND into the LC training process: (1) Pixel-Wise JND Loss (PWL) prioritizes pixel-by-pixel fidelity in reproducing JND characteristics, (2) Image-Wise JND Loss (IWL) emphasizes on overall imperceptible degradation levels, and (3) Feature-Wise JND Loss (FWL) aligns the reconstructed image features with perceptually significant features. Experimental evaluations demonstrate the effectiveness of JND integration, highlighting improvements in rate-distortion performance and visual quality, compared to baseline methods. The proposed methods add no extra complexity after training.

Related papers

• SG-JND: Semantic-Guided Just Noticeable Distortion Predictor For Image Compression [50.2496399381438]
  Just noticeable distortion (JND) represents the threshold of distortion in an image that is minimally perceptible to the human visual system. Traditional JND prediction methods only rely on pixel-level or sub-band level features. We propose a Semantic-Guided JND network to leverage semantic information for JND prediction.
  arXiv  Detail & Related papers  (2024-08-08T07:14:57Z)
• Exploring Distortion Prior with Latent Diffusion Models for Remote Sensing Image Compression [9.742764207747697]
  We propose a latent diffusion model-based remote sensing image compression (LDM-RSIC) method. In the first stage, a self-encoder learns prior from the high-quality input image. In the second stage, the prior is generated through an LDM conditioned on the decoded image of an existing learning-based image compression algorithm.
  arXiv  Detail & Related papers  (2024-06-06T11:13:44Z)
• Channel-wise Feature Decorrelation for Enhanced Learned Image Compression [16.638869231028437]
  The emerging Learned Compression (LC) replaces the traditional modules with Deep Neural Networks (DNN), which are trained end-to-end for rate-distortion performance. This paper proposes to improve compression by fully exploiting the existing DNN capacity. Three strategies are proposed and evaluated, which optimize (1) the transformation network, (2) the context model, and (3) both networks.
  arXiv  Detail & Related papers  (2024-03-16T14:30:25Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• JND-Based Perceptual Optimization For Learned Image Compression [42.822121565430926]
  We propose a JND-based perceptual quality loss for learned image compression schemes. We show that the proposed method has led to better perceptual quality than the baseline model under the same bit rate.
  arXiv  Detail & Related papers  (2023-02-25T14:49:09Z)
• Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image Denoising [50.039949798156826]
  This paper tackles the challenging problem of hyperspectral (HS) image denoising. We propose rank-enhanced low-dimensional convolution set (Re-ConvSet) We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
  arXiv  Detail & Related papers  (2022-07-09T13:35:12Z)
• Real-World Image Super-Resolution by Exclusionary Dual-Learning [98.36096041099906]
  Real-world image super-resolution is a practical image restoration problem that aims to obtain high-quality images from in-the-wild input. Deep learning-based methods have achieved promising restoration quality on real-world image super-resolution datasets. We propose Real-World image Super-Resolution by Exclusionary Dual-Learning (RWSR-EDL) to address the feature diversity in perceptual- and L1-based cooperative learning.
  arXiv  Detail & Related papers  (2022-06-06T13:28:15Z)
• Low-light Image Enhancement by Retinex Based Algorithm Unrolling and Adjustment [50.13230641857892]
  We propose a new deep learning framework for the low-light image enhancement (LIE) problem. The proposed framework contains a decomposition network inspired by algorithm unrolling, and adjustment networks considering both global brightness and local brightness sensitivity. Experiments on a series of typical LIE datasets demonstrated the effectiveness of the proposed method, both quantitatively and visually, as compared with existing methods.
  arXiv  Detail & Related papers  (2022-02-12T03:59:38Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Generalized Octave Convolutions for Learned Multi-Frequency Image Compression [20.504561050200365]
  We propose the first learned multi-frequency image compression and entropy coding approach. It is based on the recently developed octave convolutions to factorize the latents into high and low frequency (resolution) components. We show that the proposed generalized octave convolution can improve the performance of other auto-encoder-based computer vision tasks.
  arXiv  Detail & Related papers  (2020-02-24T01:35:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.