Joint End-to-End Image Compression and Denoising: Leveraging Contrastive Learning and Multi-Scale Self-ONNs

• URL: http://arxiv.org/abs/2402.05582v1
• Date: Thu, 8 Feb 2024 11:33:16 GMT
• Title: Joint End-to-End Image Compression and Denoising: Leveraging Contrastive Learning and Multi-Scale Self-ONNs
• Authors: Yuxin Xie, Li Yu, Farhad Pakdaman, Moncef Gabbouj
• Abstract summary: Noisy images are a challenge to image compression algorithms due to the inherent difficulty of compressing noise. We propose a novel method integrating a multi-scale denoiser comprising of Self Organizing Operational Neural Networks, for joint image compression and denoising.
• Score: 18.71504105967766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Noisy images are a challenge to image compression algorithms due to the inherent difficulty of compressing noise. As noise cannot easily be discerned from image details, such as high-frequency signals, its presence leads to extra bits needed for compression. Since the emerging learned image compression paradigm enables end-to-end optimization of codecs, recent efforts were made to integrate denoising into the compression model, relying on clean image features to guide denoising. However, these methods exhibit suboptimal performance under high noise levels, lacking the capability to generalize across diverse noise types. In this paper, we propose a novel method integrating a multi-scale denoiser comprising of Self Organizing Operational Neural Networks, for joint image compression and denoising. We employ contrastive learning to boost the network ability to differentiate noise from high frequency signal components, by emphasizing the correlation between noisy and clean counterparts. Experimental results demonstrate the effectiveness of the proposed method both in rate-distortion performance, and codec speed, outperforming the current state-of-the-art.

Related papers

• Transfer CLIP for Generalizable Image Denoising [11.144858989063522]
  We devise an asymmetrical encoder-decoder denoising network, which incorporates dense features including the noisy image. Experiments and comparisons conducted across diverse OOD noises, including synthetic noise, real-world sRGB noise, and low-dose CT image noise, demonstrate the superior generalization ability of our method.
  arXiv  Detail & Related papers  (2024-03-22T11:33:04Z)
• NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation [86.7260950382448]
  We propose a novel approach to correct noise for image validity, NoiseDiffusion. NoiseDiffusion performs within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss.
  arXiv  Detail & Related papers  (2024-03-13T12:32:25Z)
• On the Importance of Denoising when Learning to Compress Images [34.99683302788977]
  We propose to explicitly learn the image denoising task when training a. We leverage the Natural Image Noise dataset, which offers a wide variety of scenes captured with various ISO numbers. We show that a single model trained based on a mixture of images with variable noise levels appears to yield best-in-class results with both noisy and clean images.
  arXiv  Detail & Related papers  (2023-07-12T15:26:04Z)
• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Hierarchical Disentangled Representation for Invertible Image Denoising and Beyond [14.432771193620702]
  Inspired by a latent observation that noise tends to appear in the high-frequency part of the image, we propose a fully invertible denoising method. We decompose the noisy image into clean low-frequency and hybrid high-frequency parts with an invertible transformation. In this way, denoising is made tractable by inversely merging noiseless low and high-frequency parts.
  arXiv  Detail & Related papers  (2023-01-31T01:24:34Z)
• Optimizing Image Compression via Joint Learning with Denoising [49.83680496296047]
  High levels of noise usually exist in today's captured images due to the relatively small sensors equipped in the smartphone cameras. We propose a novel two-branch, weight-sharing architecture with plug-in feature denoisers to allow a simple and effective realization of the goal with little computational cost.
  arXiv  Detail & Related papers  (2022-07-22T04:23:01Z)
• Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis [148.16279746287452]
  We propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise. Experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-24T18:11:31Z)
• Synergy Between Semantic Segmentation and Image Denoising via Alternate Boosting [102.19116213923614]
  We propose a boosting network to perform denoising and segmentation alternately. We observe that not only denoising helps combat the drop of segmentation accuracy due to noise, but also pixel-wise semantic information boosts the capability of denoising. Experimental results show that the denoised image quality is improved substantially and the segmentation accuracy is improved to close to that of clean images.
  arXiv  Detail & Related papers  (2021-02-24T06:48:45Z)
• Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation [52.75909685172843]
  Real-world image noise removal is a long-standing yet very challenging task in computer vision. We propose a novel unified framework to deal with the noise removal and noise generation tasks. Our method learns the joint distribution of the clean-noisy image pairs.
  arXiv  Detail & Related papers  (2020-07-12T09:16:06Z)
• When deep denoising meets iterative phase retrieval [5.639904484784126]
  Conventional algorithms for retrieving the phase suffer when noise is present but display global convergence when given clean data. Here, we combine iterative methods from phase retrieval with image statistics from deep denoisers, via regularization-by-denoising. The resulting methods inherit the advantages of each approach and outperform other noise-robust phase retrieval algorithms.
  arXiv  Detail & Related papers  (2020-03-03T21:00:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.