Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence

• URL: http://arxiv.org/abs/2601.20769v2
• Date: Thu, 29 Jan 2026 03:16:47 GMT
• Title: Leveraging Second-Order Curvature for Efficient Learned Image Compression: Theory and Empirical Evidence
• Authors: Yichi Zhang, Fengqing Zhu,
• Abstract summary: We show that a second-order quasi-Newton, textSOAP, dramatically improves both training efficiency and final performance across diverse LICs.<n>We uncover a critical deployability benefit: second-order trained models exhibit significantly fewer activation and latent robustnesss.
• Score: 13.56541419560425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training learned image compression (LIC) models entails navigating a challenging optimization landscape defined by the fundamental trade-off between rate and distortion. Standard first-order optimizers, such as SGD and Adam, struggle with \emph{gradient conflicts} arising from competing objectives, leading to slow convergence and suboptimal rate-distortion performance. In this work, we demonstrate that a simple utilization of a second-order quasi-Newton optimizer, \textbf{SOAP}, dramatically improves both training efficiency and final performance across diverse LICs. Our theoretical and empirical analyses reveal that Newton preconditioning inherently resolves the intra-step and inter-step update conflicts intrinsic to the R-D objective, facilitating faster, more stable convergence. Beyond acceleration, we uncover a critical deployability benefit: second-order trained models exhibit significantly fewer activation and latent outliers. This substantially enhances robustness to post-training quantization. Together, these results establish second-order optimization, achievable as a seamless drop-in replacement of the imported optimizer, as a powerful, practical tool for advancing the efficiency and real-world readiness of LICs.

Related papers

• Bidirectional Reward-Guided Diffusion for Real-World Image Super-Resolution [79.35296000454694]
  Diffusion-based super-resolution can synthesize rich details, but models trained on synthetic paired data often fail on real-world LR images.<n>We propose Bird-SR, a reward-guided diffusion framework that formulates super-resolution as trajectory-level preference optimization.<n>Experiments on real-world SR benchmarks demonstrate that Bird-SR consistently outperforms state-of-the-art methods in perceptual quality.
  arXiv  Detail & Related papers  (2026-02-05T19:21:45Z)
• Implicit Neural Representation-Based Continuous Single Image Super Resolution: An Empirical Study [50.15623093332659]
  Implicit neural representation (INR) has become the standard approach for arbitrary-scale image super-resolution (ASSR)<n>We compare existing techniques across diverse settings and present aggregated performance results on multiple image quality metrics.<n>We examine a new loss function that penalizes intensity variations while preserving edges, textures, and finer details during training.
  arXiv  Detail & Related papers  (2026-01-25T07:09:20Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• A Simple and Effective Reinforcement Learning Method for Text-to-Image Diffusion Fine-tuning [61.403275660120606]
  Reinforcement learning (RL)-based fine-tuning has emerged as a powerful approach for aligning diffusion models with black-box objectives.<n>We propose leave-one-out PPO (LOOP), a novel RL for diffusion fine-tuning method.<n>Our results demonstrate that LOOP effectively improves diffusion models on various black-box objectives, and achieves a better balance between computational efficiency and performance.
  arXiv  Detail & Related papers  (2025-03-02T13:43:53Z)
• Balanced Rate-Distortion Optimization in Learned Image Compression [11.729071258457138]
  Learnable image compression (LIC) using deep learning architectures has seen significant advancements.<n>Standard rate-distortion (R-D) optimization often encounters imbalanced updates due to diverse gradients of the rate and distortion objectives.<n>We introduce two balanced R-D optimization strategies that adaptively adjust gradient updates to achieve more equitable improvements in both rate and distortion.
  arXiv  Detail & Related papers  (2025-02-27T15:00:43Z)
• Refining Adaptive Zeroth-Order Optimization at Ease [24.327161891577727]
  This paper introduces Refined Adaptive Zeroth-Order Optimization (R-AdaZO)<n>We first show the untapped variance reduction effect of first moment estimate on ZO gradient estimation.<n>We then refine the second moment estimate based on these variance-reduced gradient estimates to better capture the geometry of the optimization landscape.
  arXiv  Detail & Related papers  (2025-02-03T03:10:44Z)
• E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models [15.270657838960114]
  Diffusion models have established themselves as the de facto primary paradigm in visual generative modeling.<n>We present a novel end-to-end learning paradigm that establishes direct optimization from the final generated samples to initial noises.<n>Our method achieves substantial performance gains in terms of Fr'eche't Inception Distance (FID) and CLIP score, even with fewer sampling steps.
  arXiv  Detail & Related papers  (2024-12-30T16:06:31Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• FeTrIL++: Feature Translation for Exemplar-Free Class-Incremental Learning with Hill-Climbing [3.533544633664583]
  Exemplar-free class-incremental learning (EFCIL) poses significant challenges, primarily due to catastrophic forgetting. Traditional EFCIL approaches typically skew towards either model plasticity through successive fine-tuning or stability. This paper builds upon the foundational FeTrIL framework to examine the efficacy of various oversampling techniques and dynamic optimization strategies.
  arXiv  Detail & Related papers  (2024-03-12T08:34:05Z)
• MUSIC: Accelerated Convergence for Distributed Optimization With Inexact and Exact Methods [6.800113478497425]
  In this paper, we propose an accelerated framework named as MUSIC allowing each agent to perform multiple local updates and a single combination in each iteration. We equip inexact and exact distributed optimization methods into this framework, thereby developing two new algorithms that exhibit accelerated linear convergence and high communication efficiency.
  arXiv  Detail & Related papers  (2024-03-05T02:02:00Z)
• Gradient constrained sharpness-aware prompt learning for vision-language models [99.74832984957025]
  This paper targets a novel trade-off problem in generalizable prompt learning for vision-language models (VLM) By analyzing the loss landscapes of the state-of-the-art method and vanilla Sharpness-aware Minimization (SAM) based method, we conclude that the trade-off performance correlates to both loss value and loss sharpness. We propose a novel SAM-based method for prompt learning, denoted as Gradient Constrained Sharpness-aware Context Optimization (GCSCoOp)
  arXiv  Detail & Related papers  (2023-09-14T17:13:54Z)
• Nesterov Meets Optimism: Rate-Optimal Separable Minimax Optimization [108.35402316802765]
  We propose a new first-order optimization algorithm -- AcceleratedGradient-OptimisticGradient (AG-OG) Ascent. We show that AG-OG achieves the optimal convergence rate (up to a constant) for a variety of settings. We extend our algorithm to extend the setting and achieve the optimal convergence rate in both bi-SC-SC and bi-C-SC settings.
  arXiv  Detail & Related papers  (2022-10-31T17:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.