HLIC: Harmonizing Optimization Metrics in Learned Image Compression by Reinforcement Learning

• URL: http://arxiv.org/abs/2109.14863v1
• Date: Thu, 30 Sep 2021 06:01:57 GMT
• Title: HLIC: Harmonizing Optimization Metrics in Learned Image Compression by Reinforcement Learning
• Authors: Baocheng Sun, Meng Gu, Dailan He, Tongda Xu, Yan Wang, Hongwei Qin
• Abstract summary: Peak signal-to-noise ratio (PSNR) and multi-scale structural similarity (MS-SSIM) are the two most popular evaluation metrics. We propose to Harmonize optimization metrics in Learned Image Compression (HLIC) using online loss function adaptation by reinforcement learning.
• Score: 5.943388055895372
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learned image compression is making good progress in recent years. Peak signal-to-noise ratio (PSNR) and multi-scale structural similarity (MS-SSIM) are the two most popular evaluation metrics. As different metrics only reflect certain aspects of human perception, works in this field normally optimize two models using PSNR and MS-SSIM as loss function separately, which is suboptimal and makes it difficult to select the model with best visual quality or overall performance. Towards solving this problem, we propose to Harmonize optimization metrics in Learned Image Compression (HLIC) using online loss function adaptation by reinforcement learning. By doing so, we are able to leverage the advantages of both PSNR and MS-SSIM, achieving better visual quality and higher VMAF score. To our knowledge, our work is the first to explore automatic loss function adaptation for harmonizing optimization metrics in low level vision tasks like learned image compression.

Related papers

• Predicting Satisfied User and Machine Ratio for Compressed Images: A Unified Approach [58.71009078356928]
  We create a deep learning-based model to predict Satisfied User Ratio (SUR) and Satisfied Machine Ratio (SMR) of compressed images simultaneously. Experimental results indicate that the proposed model significantly outperforms state-of-the-art SUR and SMR prediction methods.
  arXiv  Detail & Related papers  (2024-12-23T11:09:30Z)
• PETALface: Parameter Efficient Transfer Learning for Low-resolution Face Recognition [54.642714288448744]
  PETALface is the first work leveraging the powers of PEFT for low resolution face recognition. We introduce two low-rank adaptation modules to the backbone, with weights adjusted based on the input image quality to account for the difference in quality for the gallery and probe images. Experiments demonstrate that the proposed method outperforms full fine-tuning on low-resolution datasets while preserving performance on high-resolution and mixed-quality datasets.
  arXiv  Detail & Related papers  (2024-12-10T18:59:45Z)
• Perceptual-Distortion Balanced Image Super-Resolution is a Multi-Objective Optimization Problem [23.833099288826045]
  Training Single-Image Super-Resolution (SISR) models using pixel-based regression losses can achieve high distortion metrics scores. However, they often results in blurry images due to insufficient recovery of high-frequency details. We propose a novel method that incorporates Multi-Objective Optimization (MOO) into the training process of SISR models to balance perceptual quality and distortion.
  arXiv  Detail & Related papers  (2024-09-05T02:14:04Z)
• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Gradient constrained sharpness-aware prompt learning for vision-language models [99.74832984957025]
  This paper targets a novel trade-off problem in generalizable prompt learning for vision-language models (VLM) By analyzing the loss landscapes of the state-of-the-art method and vanilla Sharpness-aware Minimization (SAM) based method, we conclude that the trade-off performance correlates to both loss value and loss sharpness. We propose a novel SAM-based method for prompt learning, denoted as Gradient Constrained Sharpness-aware Context Optimization (GCSCoOp)
  arXiv  Detail & Related papers  (2023-09-14T17:13:54Z)
• JND-Based Perceptual Optimization For Learned Image Compression [42.822121565430926]
  We propose a JND-based perceptual quality loss for learned image compression schemes. We show that the proposed method has led to better perceptual quality than the baseline model under the same bit rate.
  arXiv  Detail & Related papers  (2023-02-25T14:49:09Z)
• Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image Denoising [50.039949798156826]
  This paper tackles the challenging problem of hyperspectral (HS) image denoising. We propose rank-enhanced low-dimensional convolution set (Re-ConvSet) We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
  arXiv  Detail & Related papers  (2022-07-09T13:35:12Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)
• Perceptually Optimizing Deep Image Compression [53.705543593594285]
  Mean squared error (MSE) and $ell_p$ norms have largely dominated the measurement of loss in neural networks. We propose a different proxy approach to optimize image analysis networks against quantitative perceptual models.
  arXiv  Detail & Related papers  (2020-07-03T14:33:28Z)
• Saliency Driven Perceptual Image Compression [6.201592931432016]
  The paper demonstrates that the popularly used evaluations metrics such as MS-SSIM and PSNR are inadequate for judging the performance of image compression techniques. A new metric is proposed, which is learned on perceptual similarity data specific to image compression. The model not only generates images which are visually better but also gives superior performance for subsequent computer vision tasks.
  arXiv  Detail & Related papers  (2020-02-12T13:43:17Z)
• An End-to-End Joint Learning Scheme of Image Compression and Quality Enhancement with Improved Entropy Minimization [43.878329556261924]
  We propose a novel joint learning scheme of image compression and quality enhancement, called JointIQ-Net. Our proposed JointIQ-Net combines an image compression sub-network and a quality enhancement sub-network in a cascade, both of which are end-to-end trained in a combined manner within the JointIQ-Net.
  arXiv  Detail & Related papers  (2019-12-30T05:10:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.