LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization

• URL: http://arxiv.org/abs/2606.08993v1
• Date: Mon, 08 Jun 2026 03:44:20 GMT
• Title: LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization
• Abstract summary: We propose LEAF, a learning-enabled ADMM framework for accelerated convex optimization.<n>We approximate the Moreau envelope of the objective function using an Input Convex Neural Network (ICNN)<n>We show that LEAF learns a scalar-valued Moreau envelope, significantly reducing model complexity and improving data efficiency.
• Score: 1.838884769821944
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose LEAF, a learning-enabled ADMM framework for accelerated convex optimization. The key idea is to approximate the Moreau envelope of the objective function using an Input Convex Neural Network (ICNN), resulting in a learned model that preserves convexity and smoothness. This leads to the proposed Moreau Envelope Learning ADMM (MEL-ADMM) and its splitting variant sMEL-ADMM. Unlike existing approaches that learn high-dimensional operators directly, LEAF learns a scalar-valued Moreau envelope, significantly reducing model complexity and improving data efficiency. The framework accommodates a broad class of convex problems with smooth and non-smooth objectives. By embedding convexity explicitly through the ICNN architecture, the proposed approach maintains high approximation accuracy while preserving key structural properties of the optimization problem. Both MEL-ADMM and sMEL-ADMM are developed with theoretical guarantees of convergence and feasibility under the learned model. Rigorous analysis shows that the proposed methods achieve convergence rates comparable to classical ADMM while reducing per-iteration computational cost. Numerical experiments demonstrate up to an order-of-magnitude speedup over state-of-the-art solvers while maintaining low optimality gaps

Related papers

• New Hybrid Fine-Tuning Paradigm for LLMs: Algorithm Design and Convergence Analysis Framework [52.69678300445233]
  Fine-tuning Large Language Models (LLMs) typically involves either full fine-tuning, which updates all model parameters, or.<n>Efficient Fine-Tuning (PEFT) adjusts a small subset of parameters.<n>We propose a novel hybrid fine-tuning approach that jointly updates both LLMs and PEFT modules using a combination of zeroth-order and first-order optimization methods.
  arXiv  Detail & Related papers  (2026-04-10T22:39:38Z)
• Majorization-Minimization Networks for Inverse Problems: An Application to EEG Imaging [4.063392865490957]
  Inverse problems are often ill-posed and require optimization schemes with strong stability and convergence guarantees.<n>We propose a learned Majorization-Minimization (MM) framework for inverse problems within a bilevel optimization setting.<n>We learn a structured curvature majorant that governs each MM step while preserving classical MM descent guarantees.
  arXiv  Detail & Related papers  (2026-01-23T10:33:45Z)
• Learning to accelerate distributed ADMM using graph neural networks [4.309217525488745]
  We show that distributed ADMM iterations can be naturally represented within the message-passing framework of graph neural networks.<n>We train the network parameters end-to-end to minimize the final iterates error for a given problem class, while preserving the algorithm's convergence properties.
  arXiv  Detail & Related papers  (2025-09-05T17:55:22Z)
• A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning [74.80956524812714]
  We tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning. These problems are often formalized as Bi-Level optimizations (BLO) We introduce a novel perspective by turning a given BLO problem into a ii optimization, where the inner loss function becomes a smooth distribution, and the outer loss becomes an expected loss over the inner distribution.
  arXiv  Detail & Related papers  (2024-10-14T12:10:06Z)
• AA-DLADMM: An Accelerated ADMM-based Framework for Training Deep Neural Networks [1.3812010983144802]
  gradient descent (SGD) and its many variants are the widespread optimization algorithms for training deep neural networks. SGD suffers from inevitable drawbacks, including vanishing gradients, lack of theoretical guarantees, and substantial sensitivity to input. This paper proposes an Anderson Acceleration for Deep Learning ADMM (AA-DLADMM) algorithm to tackle this drawback.
  arXiv  Detail & Related papers  (2024-01-08T01:22:00Z)
• Moreau Envelope ADMM for Decentralized Weakly Convex Optimization [55.2289666758254]
  This paper proposes a proximal variant of the alternating direction method of multipliers (ADMM) for distributed optimization. The results of our numerical experiments indicate that our method is faster and more robust than widely-used approaches.
  arXiv  Detail & Related papers  (2023-08-31T14:16:30Z)
• Rényi Divergence Deep Mutual Learning [3.682680183777648]
  This paper revisits Deep Learning Mutual (DML) as a simple yet effective computing paradigm. We propose using R'enyi divergence instead of the KL divergence, which is more flexible and limited. Our empirical results demonstrate the advantage combining DML and R'enyi divergence, leading to further improvement in model generalization.
  arXiv  Detail & Related papers  (2022-09-13T04:58:35Z)
• Meta-Learning with Neural Tangent Kernels [58.06951624702086]
  We propose the first meta-learning paradigm in the Reproducing Kernel Hilbert Space (RKHS) induced by the meta-model's Neural Tangent Kernel (NTK) Within this paradigm, we introduce two meta-learning algorithms, which no longer need a sub-optimal iterative inner-loop adaptation as in the MAML framework. We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
  arXiv  Detail & Related papers  (2021-02-07T20:53:23Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)
• Optimization-driven Machine Learning for Intelligent Reflecting Surfaces Assisted Wireless Networks [82.33619654835348]
  Intelligent surface (IRS) has been employed to reshape the wireless channels by controlling individual scattering elements' phase shifts. Due to the large size of scattering elements, the passive beamforming is typically challenged by the high computational complexity. In this article, we focus on machine learning (ML) approaches for performance in IRS-assisted wireless networks.
  arXiv  Detail & Related papers  (2020-08-29T08:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.