Differentiable JPEG: The Devil is in the Details

• URL: http://arxiv.org/abs/2309.06978v4
• Date: Fri, 22 Dec 2023 14:16:59 GMT
• Title: Differentiable JPEG: The Devil is in the Details
• Authors: Christoph Reich, Biplob Debnath, Deep Patel, Srimat Chakradhar
• Abstract summary: We propose a novel diff. JPEG approach, overcoming previous limitations. Our approach is differentiable w.r.t. the input image, the JPEG quality, the quantization tables, and the color conversion parameters. Our proposed diff. JPEG resembles the (non-diff.) reference implementation best, significantly surpassing the recent-best diff. approach by $3.47$dB (PSNR) on average.
• Score: 2.246961121930528
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: JPEG remains one of the most widespread lossy image coding methods. However, the non-differentiable nature of JPEG restricts the application in deep learning pipelines. Several differentiable approximations of JPEG have recently been proposed to address this issue. This paper conducts a comprehensive review of existing diff. JPEG approaches and identifies critical details that have been missed by previous methods. To this end, we propose a novel diff. JPEG approach, overcoming previous limitations. Our approach is differentiable w.r.t. the input image, the JPEG quality, the quantization tables, and the color conversion parameters. We evaluate the forward and backward performance of our diff. JPEG approach against existing methods. Additionally, extensive ablations are performed to evaluate crucial design choices. Our proposed diff. JPEG resembles the (non-diff.) reference implementation best, significantly surpassing the recent-best diff. approach by $3.47$dB (PSNR) on average. For strong compression rates, we can even improve PSNR by $9.51$dB. Strong adversarial attack results are yielded by our diff. JPEG, demonstrating the effective gradient approximation. Our code is available at https://github.com/necla-ml/Diff-JPEG.

Related papers

• Three Forensic Cues for JPEG AI Images [7.7834147791981305]
  We propose three cues for forensic algorithms for JPEG AI. First, we show that the JPEG AI preprocessing introduces correlations in color channels that do not occur in uncompressed images. Second, we show that repeated compression of JPEG AI images leads to diminishing distortion differences. Third, we show that the quantization of JPEG AI images in the latent space can be used to distinguish real images with JPEG AI compression from synthetically generated images.
  arXiv  Detail & Related papers  (2025-04-04T05:38:30Z)
• Compression-Aware One-Step Diffusion Model for JPEG Artifact Removal [56.307484956135355]
  CODiff is a compression-aware one-step diffusion model for JPEG artifact removal. We propose a dual learning strategy that combines explicit and implicit learning. Results demonstrate that CODiff surpasses recent leading methods in both quantitative and visual quality metrics.
  arXiv  Detail & Related papers  (2025-02-14T02:46:27Z)
• JPEG Inspired Deep Learning [4.958744940097937]
  Well-crafted JPEG compression can actually improve the performance of deep learning (DL) We propose JPEG-DL, a novel DL framework that prepends any underlying DNN architecture with a trainable JPEG compression layer.
  arXiv  Detail & Related papers  (2024-10-09T17:23:54Z)
• Unified learning-based lossy and lossless JPEG recompression [15.922937139019547]
  We propose a unified lossly and lossless JPEG recompression framework, which consists of learned quantization table and Markovian hierarchical variational autoencoders. Experiments show that our method can achieve arbitrarily low distortion when the JPEG is close to the upper bound.
  arXiv  Detail & Related papers  (2023-12-05T12:07:27Z)
• Learned Lossless Compression for JPEG via Frequency-Domain Prediction [50.20577108662153]
  We propose a novel framework for learned lossless compression of JPEG images. To enable learning in the frequency domain, DCT coefficients are partitioned into groups to utilize implicit local redundancy. An autoencoder-like architecture is designed based on the weight-shared blocks to realize entropy modeling of grouped DCT coefficients.
  arXiv  Detail & Related papers  (2023-03-05T13:15:28Z)
• High-Perceptual Quality JPEG Decoding via Posterior Sampling [13.238373528922194]
  We propose a different paradigm for JPEG artifact correction. We aim to obtain sharp, detailed and visually reconstructed images, while being consistent with the compressed input. Our solution offers a diverse set of plausible and fast reconstructions for a given input with perfect consistency.
  arXiv  Detail & Related papers  (2022-11-21T19:47:59Z)
• Practical Learned Lossless JPEG Recompression with Multi-Level Cross-Channel Entropy Model in the DCT Domain [10.655855413391324]
  We propose a deep learning based JPEG recompression method that operates on DCT domain. Experiments show that our method achieves state-of-the-art performance compared with traditional JPEG recompression methods.
  arXiv  Detail & Related papers  (2022-03-30T14:36:13Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Towards Flexible Blind JPEG Artifacts Removal [73.46374658847675]
  We propose a flexible blind convolutional neural network, namely FBCNN, that can predict the adjustable quality factor to control the trade-off between artifacts removal and details preservation. Our proposed FBCNN achieves favorable performance against state-of-the-art methods in terms of both quantitative metrics and visual quality.
  arXiv  Detail & Related papers  (2021-09-29T17:12:10Z)
• Towards Robust Data Hiding Against (JPEG) Compression: A Pseudo-Differentiable Deep Learning Approach [78.05383266222285]
  It is still an open challenge to achieve the goal of data hiding that can be against these compressions. Deep learning has shown large success in data hiding, while non-differentiability of JPEG makes it challenging to train a deep pipeline for improving robustness against lossy compression. In this work, we propose a simple yet effective approach to address all the above limitations at once.
  arXiv  Detail & Related papers  (2020-12-30T12:30:09Z)
• Learning to Improve Image Compression without Changing the Standard Decoder [100.32492297717056]
  We propose learning to improve the encoding performance with the standard decoder. Specifically, a frequency-domain pre-editing method is proposed to optimize the distribution of DCT coefficients. We do not modify the JPEG decoder and therefore our approach is applicable when viewing images with the widely used standard JPEG decoder.
  arXiv  Detail & Related papers  (2020-09-27T19:24:42Z)
• Quantization Guided JPEG Artifact Correction [69.04777875711646]
  We develop a novel architecture for artifact correction using the JPEG files quantization matrix. This allows our single model to achieve state-of-the-art performance over models trained for specific quality settings.
  arXiv  Detail & Related papers  (2020-04-17T00:10:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.