Riemannian Lyapunov Optimizer: A Unified Framework for Optimization

• URL: http://arxiv.org/abs/2601.22284v1
• Date: Thu, 29 Jan 2026 20:00:25 GMT
• Title: Riemannian Lyapunov Optimizer: A Unified Framework for Optimization
• Authors: Yixuan Wang, Omkar Sudhir Patil, Warren E. Dixon,
• Abstract summary: We introduce a family of optimization algorithms that unifies classic algorithms within one geometric framework.<n>RLOs bridge control theory and modern machine learning optimization, providing a unified language and a systematic toolkit.
• Score: 6.493476506951333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Riemannian Lyapunov Optimizers (RLOs), a family of optimization algorithms that unifies classic optimizers within one geometric framework. Unlike heuristic improvements to existing optimizers, RLOs are systematically derived from a novel control-theoretic framework that reinterprets optimization as an extended state discrete-time controlled dynamical system on a Riemannian parameter manifold. Central to this framework is the identification of a Normally Attracting Invariant Manifold (NAIM), which organizes training dynamics into two distinct stages: rapid alignment of the speed state to a target graph, followed by controlled evolution within it. We formalize this by constructing a strict Lyapunov function that certifies convergence to a target manifold. This perspective yields a constructive ``optimizer generator" that not only recovers classic algorithms but enables the principled design of RLOs. We validate our theory via geometric diagnostics and demonstrate that grounding optimizer design in control theory yields state-of-the-art performance in large-scale benchmarks. Overall, RLOs bridge control theory and modern machine learning optimization, providing a unified language and a systematic toolkit for designing stable, effective optimizers.

Related papers

• ODELoRA: Training Low-Rank Adaptation by Solving Ordinary Differential Equations [54.886931928255564]
  Low-rank adaptation (LoRA) has emerged as a widely adopted parameter-efficient fine-tuning method in deep transfer learning.<n>We propose a novel continuous-time optimization dynamic for LoRA factor matrices in the form of an ordinary differential equation (ODE)<n>We show that ODELoRA achieves stable feature learning, a property that is crucial for training deep neural networks at different scales of problem dimensionality.
  arXiv  Detail & Related papers  (2026-02-07T10:19:36Z)
• Deep Unfolding: Recent Developments, Theory, and Design Guidelines [99.63555420898554]
  This article provides a tutorial-style overview of deep unfolding, a framework that transforms optimization algorithms into structured, trainable ML architectures.<n>We review the foundations of optimization for inference and for learning, introduce four representative design paradigms for deep unfolding, and discuss the distinctive training schemes that arise from their iterative nature.
  arXiv  Detail & Related papers  (2025-12-03T13:16:35Z)
• Beyond Heuristics: Globally Optimal Configuration of Implicit Neural Representations [8.864909622103388]
  Implicit Neural Representations (INRs) have emerged as a transformative paradigm in signal processing and computer vision.<n>But their effectiveness is limited by the absence of principled strategies for optimal configuration.<n>This work introduces OptiINR, the first unified framework that formulates INR configuration as a rigorous optimization problem.
  arXiv  Detail & Related papers  (2025-09-27T05:45:51Z)
• LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters [43.04933165005961]
  We present a novel framework for Low-Rank Adaptation (LoRA)<n>LoRA geometrically treats low-rank adapters by optimizing them directly on the fixed-rank manifold.<n>Our framework integrates three key components to achieve this.
  arXiv  Detail & Related papers  (2025-07-16T11:17:12Z)
• Divergence Minimization Preference Optimization for Diffusion Model Alignment [66.31417479052774]
  Divergence Minimization Preference Optimization (DMPO) is a principled method for aligning diffusion models by minimizing reverse KL divergence.<n>DMPO can consistently outperform or match existing techniques across different base models and test sets.
  arXiv  Detail & Related papers  (2025-07-10T07:57:30Z)
• Return of the Latent Space COWBOYS: Re-thinking the use of VAEs for Bayesian Optimisation of Structured Spaces [13.38402522324075]
  We propose a decoupled approach that trains a generative model and a Gaussian Process (GP) surrogate separately, then combines them via a simple yet principled Bayesian update rule.<n>We show that our decoupled approach improves our ability to identify high-potential candidates in molecular optimisation problems under constrained evaluation budgets.
  arXiv  Detail & Related papers  (2025-07-05T05:53:04Z)
• Iterate to Accelerate: A Unified Framework for Iterative Reasoning and Feedback Convergence [0.0]
  We introduce a unified framework for iterative reasoning that leverages non-Euclidean geometry via Bregman divergences, higher-order operator averaging, and adaptive feedback mechanisms.<n>Our analysis establishes that, under mild smoothness and contractivity assumptions, a generalized update scheme not only unifies classical methods such as mirror descent and dynamic programming but also captures modern chain-of-thought reasoning processes in large language models.
  arXiv  Detail & Related papers  (2025-02-06T05:24:35Z)
• Accelerated Federated Learning with Decoupled Adaptive Optimization [53.230515878096426]
  federated learning (FL) framework enables clients to collaboratively learn a shared model while keeping privacy of training data on clients. Recently, many iterations efforts have been made to generalize centralized adaptive optimization methods, such as SGDM, Adam, AdaGrad, etc., to federated settings. This work aims to develop novel adaptive optimization methods for FL from the perspective of dynamics of ordinary differential equations (ODEs)
  arXiv  Detail & Related papers  (2022-07-14T22:46:43Z)
• Optimization on manifolds: A symplectic approach [127.54402681305629]
  We propose a dissipative extension of Dirac's theory of constrained Hamiltonian systems as a general framework for solving optimization problems. Our class of (accelerated) algorithms are not only simple and efficient but also applicable to a broad range of contexts.
  arXiv  Detail & Related papers  (2021-07-23T13:43:34Z)
• Optimization-Inspired Learning with Architecture Augmentations and Control Mechanisms for Low-Level Vision [74.9260745577362]
  This paper proposes a unified optimization-inspired learning framework to aggregate Generative, Discriminative, and Corrective (GDC) principles. We construct three propagative modules to effectively solve the optimization models with flexible combinations. Experiments across varied low-level vision tasks validate the efficacy and adaptability of GDC.
  arXiv  Detail & Related papers  (2020-12-10T03:24:53Z)
• EOS: a Parallel, Self-Adaptive, Multi-Population Evolutionary Algorithm for Constrained Global Optimization [68.8204255655161]
  EOS is a global optimization algorithm for constrained and unconstrained problems of real-valued variables. It implements a number of improvements to the well-known Differential Evolution (DE) algorithm. Results prove that EOSis capable of achieving increased performance compared to state-of-the-art single-population self-adaptive DE algorithms.
  arXiv  Detail & Related papers  (2020-07-09T10:19:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.