Mirror Descent Using the Tempesta Generalized Multi-parametric Logarithms

• URL: http://arxiv.org/abs/2506.13984v1
• Date: Sun, 08 Jun 2025 17:48:44 GMT
• Title: Mirror Descent Using the Tempesta Generalized Multi-parametric Logarithms
• Authors: Andrzej Cichocki,
• Abstract summary: We develop a wide class Mirror Descent (MD) algorithms, which play a key role in machine learning.<n>We exploit the Bregman divergence with the Tempesta multi-parametric deformation logarithm as a link function.<n>We generate a new wide and flexible family of MD and mirror-less MD updates.
• Score: 14.572732893433825
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we develop a wide class Mirror Descent (MD) algorithms, which play a key role in machine learning. For this purpose we formulated the constrained optimization problem, in which we exploits the Bregman divergence with the Tempesta multi-parametric deformation logarithm as a link function. This link function called also mirror function defines the mapping between the primal and dual spaces and is associated with a very-wide (in fact, theoretically infinite) class of generalized trace-form entropies. In order to derive novel MD updates, we estimate generalized exponential function, which closely approximates the inverse of the multi-parametric Tempesta generalized logarithm. The shape and properties of the Tempesta logarithm and its inverse-deformed exponential functions can be tuned by several hyperparameters. By learning these hyperparameters, we can adapt to distribution or geometry of training data, and we can adjust them to achieve desired properties of MD algorithms. The concept of applying multi-parametric logarithms allow us to generate a new wide and flexible family of MD and mirror-less MD updates.

Related papers

• Improved Stochastic Optimization of LogSumExp [2.8547553943343797]
  We propose a novel approximation to LogSumExp that can be efficiently optimized using gradient methods.<n>The accuracy of the approximation is controlled by a tunable parameter and can be made arbitrarily small.<n> Experiments in DRO and continuous optimal transport demonstrate the advantages of our approach.
  arXiv  Detail & Related papers  (2025-09-29T15:03:55Z)
• Sequential-Parallel Duality in Prefix Scannable Models [68.39855814099997]
  Recent developments have given rise to various models, such as Gated Linear Attention (GLA) and Mamba.<n>This raises a natural question: can we characterize the full class of neural sequence models that support near-constant-time parallel evaluation and linear-time, constant-space sequential inference?
  arXiv  Detail & Related papers  (2025-06-12T17:32:02Z)
• Mirror Descent and Novel Exponentiated Gradient Algorithms Using Trace-Form Entropies and Deformed Logarithms [14.283977131819285]
  We propose and investigate a class of Mirror Descent updates (MD) and associated novel Generalized Exponentiated Gradient (GEG) algorithms.<n>The proposed algorithms can be considered as extension of entropic MD and generalization of multiplicative updates.
  arXiv  Detail & Related papers  (2025-03-11T10:50:07Z)
• Generalized Exponentiated Gradient Algorithms Using the Euler Two-Parameter Logarithm [14.572732893433825]
  We propose and investigate a new class of Generalized Exponentiated Gradient (GEG) algorithms using Mirror Descent (MD) approaches.<n>The concept of generalized entropies and associated deformed logarithms provide deeper insight into novel gradient descent updates.
  arXiv  Detail & Related papers  (2025-02-21T11:05:04Z)
• Meta-Learning Adversarial Bandit Algorithms [55.72892209124227]
  We study online meta-learning with bandit feedback. We learn to tune online mirror descent generalization (OMD) with self-concordant barrier regularizers.
  arXiv  Detail & Related papers  (2023-07-05T13:52:10Z)
• Adaptive Log-Euclidean Metrics for SPD Matrix Learning [73.12655932115881]
  We propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM) The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks.
  arXiv  Detail & Related papers  (2023-03-26T18:31:52Z)
• The Generalization Error of Stochastic Mirror Descent on Over-Parametrized Linear Models [37.6314945221565]
  Deep networks are known to generalize well to unseen data. Regularization properties ensure interpolating solutions with "good" properties are found. We present simulation results that validate the theory and introduce two data models.
  arXiv  Detail & Related papers  (2023-02-18T22:23:42Z)
• Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processes [80.89852729380425]
  We propose the first computationally efficient algorithm that achieves the nearly minimax optimal regret $tilde O(dsqrtH3K)$. Our work provides a complete answer to optimal RL with linear MDPs, and the developed algorithm and theoretical tools may be of independent interest.
  arXiv  Detail & Related papers  (2022-12-12T18:58:59Z)
• Stochastic Mirror Descent in Average Ensemble Models [38.38572705720122]
  The mirror descent (SMD) is a general class of training algorithms, which includes the celebrated gradient descent (SGD) as a special case. In this paper we explore the performance of the mirror potential algorithm on mean-field ensemble models.
  arXiv  Detail & Related papers  (2022-10-27T11:04:00Z)
• Implicit Regularization Properties of Variance Reduced Stochastic Mirror Descent [7.00422423634143]
  We prove that the discrete VRSMD estimator sequence converges to the minimum mirror interpolant in the linear regression. We derive a model estimation accuracy result in the setting when the true model is sparse.
  arXiv  Detail & Related papers  (2022-04-29T19:37:24Z)
• ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks [86.37110868126548]
  In this work, we make use of deep residual neural networks to solve the non-stationary ODE (flow equation) based on a Euler's discretization scheme. We illustrate these ideas on diverse registration problems of 3D shapes under complex topology-preserving transformations.
  arXiv  Detail & Related papers  (2021-02-16T04:07:13Z)
• From Sets to Multisets: Provable Variational Inference for Probabilistic Integer Submodular Models [82.95892656532696]
  Submodular functions have been studied extensively in machine learning and data mining. In this work, we propose a continuous DR-submodular extension for integer submodular functions. We formulate a new probabilistic model which is defined through integer submodular functions.
  arXiv  Detail & Related papers  (2020-06-01T22:20:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.