Learning context-aware adaptive solvers to accelerate quadratic programming

• URL: http://arxiv.org/abs/2211.12443v1
• Date: Tue, 22 Nov 2022 17:52:36 GMT
• Title: Learning context-aware adaptive solvers to accelerate quadratic programming
• Abstract summary: CA-ADMM learns to adaptively adjust $rho$ to accelerate ADMM. We show that CA-ADMM effectively generalizes to unseen QP problems with different sizes and classes.
• Score: 11.172382217477129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convex quadratic programming (QP) is an important sub-field of mathematical optimization. The alternating direction method of multipliers (ADMM) is a successful method to solve QP. Even though ADMM shows promising results in solving various types of QP, its convergence speed is known to be highly dependent on the step-size parameter $\rho$. Due to the absence of a general rule for setting $\rho$, it is often tuned manually or heuristically. In this paper, we propose CA-ADMM (Context-aware Adaptive ADMM)) which learns to adaptively adjust $\rho$ to accelerate ADMM. CA-ADMM extracts the spatio-temporal context, which captures the dependency of the primal and dual variables of QP and their temporal evolution during the ADMM iterations. CA-ADMM chooses $\rho$ based on the extracted context. Through extensive numerical experiments, we validated that CA-ADMM effectively generalizes to unseen QP problems with different sizes and classes (i.e., having different QP parameter structures). Furthermore, we verified that CA-ADMM could dynamically adjust $\rho$ considering the stage of the optimization process to accelerate the convergence speed further.

Related papers

• Learning Over-Relaxation Policies for ADMM with Convergence Guarantees [0.1784233255402269]
  We propose learning online updates of the relaxation parameter to improve performance on problem classes of interest.<n>This choice is computationally attractive in OSQP-like architectures, since adapting relaxation does not trigger the matrixizations associated with penalty updates.<n>We show that the resulting learned policies improve both iteration count and wall-clock time over baseline OSQP.
  arXiv  Detail & Related papers  (2026-04-29T17:45:52Z)
• Taming Score-Based Denoisers in ADMM: A Convergent Plug-and-Play Framework [8.895254408079316]
  We develop ADMM plug-and-play (ADMM) that embeds a three-stage denoiser into ADMM.<n>Experiments on a range inverse problems demonstrate that our method consistently improves quality over a variety baselines solution.
  arXiv  Detail & Related papers  (2026-03-10T23:55:00Z)
• Learning to accelerate distributed ADMM using graph neural networks [4.309217525488745]
  We show that distributed ADMM iterations can be naturally represented within the message-passing framework of graph neural networks.<n>We train the network parameters end-to-end to minimize the final iterates error for a given problem class, while preserving the algorithm's convergence properties.
  arXiv  Detail & Related papers  (2025-09-05T17:55:22Z)
• Distributed Primal-Dual Algorithms: Unification, Connections, and Insights [5.87788855821817]
  We study primal-dual algorithms for general empirical risk minimization problems in distributed settings. We show that both classes of algorithms can be transformed into a unified update form that involves only primal and dual variables.
  arXiv  Detail & Related papers  (2025-02-01T15:56:11Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• Optimizing ADMM and Over-Relaxed ADMM Parameters for Linear Quadratic Problems [32.04687753889809]
  Alternating Direction Method of Multipliers (ADMM) has gained significant attention across a broad spectrum of machine learning applications. We propose a general approach to optimize the value of penalty parameter, followed by a novel closed-form formula to compute the optimal relaxation parameter. We then experimentally validate our parameter selection methods through random instantiations and diverse imaging applications.
  arXiv  Detail & Related papers  (2024-01-01T04:01:40Z)
• Moreau Envelope ADMM for Decentralized Weakly Convex Optimization [55.2289666758254]
  This paper proposes a proximal variant of the alternating direction method of multipliers (ADMM) for distributed optimization. The results of our numerical experiments indicate that our method is faster and more robust than widely-used approaches.
  arXiv  Detail & Related papers  (2023-08-31T14:16:30Z)
• Learning Regionally Decentralized AC Optimal Power Flows with ADMM [16.843799157160063]
  This paper studies how machine learning may help in speeding up the convergence of ADMM for solving AC-OPF. It proposes a novel decentralized machine-learning approach, namely ML-ADMM, where each agent uses deep learning to learn the consensus parameters on the coupling branches.
  arXiv  Detail & Related papers  (2022-05-08T05:30:35Z)
• A Framework of Inertial Alternating Direction Method of Multipliers for Non-Convex Non-Smooth Optimization [17.553531291690025]
  We propose an algorithmic framework dubbed alternating methods of multipliers (iADMM) for solving a class of non nonsmooth multiblock composite problems. Our framework employs the general-major surrogateization (MM) principle to update each block of variables to unify the convergence analysis of previous ADMM schemes.
  arXiv  Detail & Related papers  (2021-02-10T13:55:28Z)
• Meta-Learning with Adaptive Hyperparameters [55.182841228303225]
  We focus on a complementary factor in MAML framework, inner-loop optimization (or fast adaptation) We propose a new weight update rule that greatly enhances the fast adaptation process.
  arXiv  Detail & Related papers  (2020-10-31T08:05:34Z)
• Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [113.52575069030192]
  Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center. We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes. A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
  arXiv  Detail & Related papers  (2020-10-02T10:41:59Z)
• Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM [52.12831959365598]
  We propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM, leverages the worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM) Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed.
  arXiv  Detail & Related papers  (2020-09-14T14:18:19Z)
• Alternating Direction Method of Multipliers for Quantization [15.62692130672419]
  We study the performance of the Alternating Direction Method of Multipliers for Quantization ($texttADMM-Q$) algorithm. We develop a few variants of $texttADMM-Q$ that can handle inexact update rules. We empirically evaluate the efficacy of our proposed approaches.
  arXiv  Detail & Related papers  (2020-09-08T01:58:02Z)
• Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization [110.52708815647613]
  In this paper, we propose a faster alternating direction of multipliers (ADMM) for non-integrated optimization by using a new path, called SPADMM. We prove that the SPADMM achieves a-breaking first-order differential oracle estimator (IFO) for finding a record of an IFO. Our theoretical analysis shows that the online SPIDER-ADMM has the IFOFO(epsilon) by a factor of $mathcalO(n1)$.
  arXiv  Detail & Related papers  (2020-08-04T02:59:42Z)
• Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning [66.18202188565922]
  We propose a communication-efficient decentralized machine learning (ML) algorithm, coined QGADMM (QGADMM) We develop a novel quantization method to adaptively adjust modelization levels and their probabilities, while proving the convergence of QGADMM for convex functions.
  arXiv  Detail & Related papers  (2019-10-23T10:47:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.