Towards a Principled Learning Rate Adaptation for Natural Evolution Strategies

• URL: http://arxiv.org/abs/2112.10680v1
• Date: Mon, 22 Nov 2021 13:20:12 GMT
• Title: Towards a Principled Learning Rate Adaptation for Natural Evolution Strategies
• Authors: Masahiro Nomura, Isao Ono
• Abstract summary: We propose a new learning rate adaptation mechanism for Natural Evolution Strategies (NES) The proposed mechanism makes it possible to set a high learning rate for problems that are relatively easy to optimize. The experimental evaluations on unimodal and multimodal functions demonstrate that the proposed mechanism works properly depending on a search situation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural Evolution Strategies (NES) is a promising framework for black-box continuous optimization problems. NES optimizes the parameters of a probability distribution based on the estimated natural gradient, and one of the key parameters affecting the performance is the learning rate. We argue that from the viewpoint of the natural gradient method, the learning rate should be determined according to the estimation accuracy of the natural gradient. To do so, we propose a new learning rate adaptation mechanism for NES. The proposed mechanism makes it possible to set a high learning rate for problems that are relatively easy to optimize, which results in speeding up the search. On the other hand, in problems that are difficult to optimize (e.g., multimodal functions), the proposed mechanism makes it possible to set a conservative learning rate when the estimation accuracy of the natural gradient seems to be low, which results in the robust and stable search. The experimental evaluations on unimodal and multimodal functions demonstrate that the proposed mechanism works properly depending on a search situation and is effective over the existing method, i.e., using the fixed learning rate.

Related papers

• Gradient-Variation Online Learning under Generalized Smoothness [56.38427425920781]
  gradient-variation online learning aims to achieve regret guarantees that scale with variations in gradients of online functions. Recent efforts in neural network optimization suggest a generalized smoothness condition, allowing smoothness to correlate with gradient norms. We provide the applications for fast-rate convergence in games and extended adversarial optimization.
  arXiv  Detail & Related papers  (2024-08-17T02:22:08Z)
• Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses [5.052293146674794]
  It is known that the standard descent (SGD) optimization method, as well as accelerated and adaptive SGD optimization methods such as the Adam fail to converge if the learning rates do not converge to zero. In this work we propose and study a learning-rate-adaptive approach for SGD optimization methods in which the learning rate is adjusted based on empirical estimates.
  arXiv  Detail & Related papers  (2024-06-20T14:07:39Z)
• Adaptive Robust Learning using Latent Bernoulli Variables [50.223140145910904]
  We present an adaptive approach for learning from corrupted training sets. We identify corrupted non-corrupted samples with latent Bernoulli variables. The resulting problem is solved via variational inference.
  arXiv  Detail & Related papers  (2023-12-01T13:50:15Z)
• Scalable Bayesian Meta-Learning through Generalized Implicit Gradients [64.21628447579772]
  Implicit Bayesian meta-learning (iBaML) method broadens the scope of learnable priors, but also quantifies the associated uncertainty. Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one.
  arXiv  Detail & Related papers  (2023-03-31T02:10:30Z)
• Fast and Correct Gradient-Based Optimisation for Probabilistic Programming via Smoothing [0.0]
  We study the foundations of variational inference, which frames posterior inference as an optimisation problem. We endow our language with both a measurable and a smoothed (approximate) value semantics. We show that our approach has a similar convergence as a key competitor, but is simpler, faster, and attains orders of magnitude reduction in work-normalised variance.
  arXiv  Detail & Related papers  (2023-01-09T15:12:45Z)
• Regret-Aware Black-Box Optimization with Natural Gradients, Trust-Regions and Entropy Control [17.430247457941284]
  Most successful black-boxs, such as CMA-ES, use rankings of the individual samples to obtain a new search distribution. While these algorithms typically produce a high-quality mean estimate of the search distribution, the produced samples can have poor quality as these algorithms are ignorant of the regret. In contrast, the Relative Entropy Search (MORE) algorithm directly optimize the expected fitness function without the use of rankings.
  arXiv  Detail & Related papers  (2022-05-24T16:25:15Z)
• A Boosting Approach to Reinforcement Learning [59.46285581748018]
  We study efficient algorithms for reinforcement learning in decision processes whose complexity is independent of the number of states. We give an efficient algorithm that is capable of improving the accuracy of such weak learning methods.
  arXiv  Detail & Related papers  (2021-08-22T16:00:45Z)
• Meta-Regularization: An Approach to Adaptive Choice of the Learning Rate in Gradient Descent [20.47598828422897]
  We propose textit-Meta-Regularization, a novel approach for the adaptive choice of the learning rate in first-order descent methods. Our approach modifies the objective function by adding a regularization term, and casts the joint process parameters.
  arXiv  Detail & Related papers  (2021-04-12T13:13:34Z)
• Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework [100.36569795440889]
  This work is on the iteration of zero-th-order (ZO) optimization which does not require first-order information. We show that with a graceful design in coordinate importance sampling, the proposed ZO optimization method is efficient both in terms of complexity as well as as function query cost.
  arXiv  Detail & Related papers  (2020-12-21T17:29:58Z)
• Adaptive Gradient Method with Resilience and Momentum [120.83046824742455]
  We propose an Adaptive Gradient Method with Resilience and Momentum (AdaRem) AdaRem adjusts the parameter-wise learning rate according to whether the direction of one parameter changes in the past is aligned with the direction of the current gradient. Our method outperforms previous adaptive learning rate-based algorithms in terms of the training speed and the test error.
  arXiv  Detail & Related papers  (2020-10-21T14:49:00Z)
• Automatic, Dynamic, and Nearly Optimal Learning Rate Specification by Local Quadratic Approximation [7.386152866234369]
  In deep learning tasks, the learning rate determines the update step size in each iteration. We propose a novel optimization method based on local quadratic approximation (LQA)
  arXiv  Detail & Related papers  (2020-04-07T10:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.