CMA-ES with Adaptive Reevaluation for Multiplicative Noise

• URL: http://arxiv.org/abs/2405.11471v1
• Date: Sun, 19 May 2024 07:42:10 GMT
• Title: CMA-ES with Adaptive Reevaluation for Multiplicative Noise
• Authors: Kento Uchida, Kenta Nishihara, Shinichi Shirakawa,
• Abstract summary: We develop the reevaluation adaptation CMA-ES (RA-CMA-ES), which computes two update directions using half of the evaluations and adapts the number of reevaluations on the estimated correlation of those two update directions. The numerical simulation shows that the RA-CMA-ES outperforms the comparative method under multiplicative noise.
• Score: 1.3108652488669732
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The covariance matrix adaptation evolution strategy (CMA-ES) is a powerful optimization method for continuous black-box optimization problems. Several noise-handling methods have been proposed to bring out the optimization performance of the CMA-ES on noisy objective functions. The adaptations of the population size and the learning rate are two major approaches that perform well under additive Gaussian noise. The reevaluation technique is another technique that evaluates each solution multiple times. In this paper, we discuss the difference between those methods from the perspective of stochastic relaxation that considers the maximization of the expected utility function. We derive that the set of maximizers of the noise-independent utility, which is used in the reevaluation technique, certainly contains the optimal solution, while the noise-dependent utility, which is used in the population size and leaning rate adaptations, does not satisfy it under multiplicative noise. Based on the discussion, we develop the reevaluation adaptation CMA-ES (RA-CMA-ES), which computes two update directions using half of the evaluations and adapts the number of reevaluations based on the estimated correlation of those two update directions. The numerical simulation shows that the RA-CMA-ES outperforms the comparative method under multiplicative noise, maintaining competitive performance under additive noise.

Related papers

• Gradient Normalization with(out) Clipping Ensures Convergence of Nonconvex SGD under Heavy-Tailed Noise with Improved Results [60.92029979853314]
  This paper investigates Gradient Normalization without (NSGDC) its gradient reduction variant (NSGDC-VR) We present significant improvements in the theoretical results for both algorithms.
  arXiv  Detail & Related papers  (2024-10-21T22:40:42Z)
• An Adaptive Re-evaluation Method for Evolution Strategy under Additive Noise [3.92625489118339]
  We propose a novel method to adaptively choose the optimal re-evaluation number for function values corrupted by additive Gaussian white noise. We experimentally compare our method to the state-of-the-art noise-handling methods for CMA-ES on a set of artificial test functions.
  arXiv  Detail & Related papers  (2024-09-25T09:10:21Z)
• CMA-ES with Learning Rate Adaptation [8.109613242730163]
  This study explores the impact of learning rate on the CMA-ES performance and demonstrates the necessity of a small learning rate. We develop a novel learning rate adaptation mechanism for the CMA-ES that maintains a constant signal-to-noise ratio. The results show that the CMA-ES with the proposed learning rate adaptation works well for multimodal and/or noisy problems without extremely expensive learning rate tuning.
  arXiv  Detail & Related papers  (2024-01-29T04:16:22Z)
• Adaptive Noise Covariance Estimation under Colored Noise using Dynamic Expectation Maximization [1.550120821358415]
  We introduce a novel brain-inspired algorithm that accurately estimates the NCM for dynamic systems subjected to colored noise. We mathematically prove that our NCM estimator converges to the global optimum of this free energy objective. Notably, we show that our method outperforms the best baseline (Variational Bayes) in joint noise and state estimation for high colored noise.
  arXiv  Detail & Related papers  (2023-08-15T14:21:53Z)
• CMA-ES with Learning Rate Adaptation: Can CMA-ES with Default Population Size Solve Multimodal and Noisy Problems? [9.114392580988552]
  We investigate whether the CMA-ES with default population size can solve multimodal and noisy problems. We develop a novel learning rate adaptation mechanism for the CMA-ES, such that the learning rate is adapted so as to maintain a constant signal-to-noise ratio. The results demonstrate that, when the proposed learning rate adaptation is used, the CMA-ES with default population size works well on multimodal and/or noisy problems.
  arXiv  Detail & Related papers  (2023-04-07T04:36:27Z)
• Fast and efficient speech enhancement with variational autoencoders [0.0]
  Unsupervised speech enhancement based on variational autoencoders has shown promising performance compared with the commonly used supervised methods. We propose a new approach based on Langevin dynamics that generates multiple sequences of samples and comes with a total variation-based regularization to incorporate temporal correlations of latent vectors. Our experiments demonstrate that the developed framework makes an effective compromise between computational efficiency and enhancement quality, and outperforms existing methods.
  arXiv  Detail & Related papers  (2022-11-02T09:52:13Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• Multi-objective hyperparameter optimization with performance uncertainty [62.997667081978825]
  This paper presents results on multi-objective hyperparameter optimization with uncertainty on the evaluation of Machine Learning algorithms. We combine the sampling strategy of Tree-structured Parzen Estimators (TPE) with the metamodel obtained after training a Gaussian Process Regression (GPR) with heterogeneous noise. Experimental results on three analytical test functions and three ML problems show the improvement over multi-objective TPE and GPR.
  arXiv  Detail & Related papers  (2022-09-09T14:58:43Z)
• Momentum Accelerates the Convergence of Stochastic AUPRC Maximization [80.8226518642952]
  We study optimization of areas under precision-recall curves (AUPRC), which is widely used for imbalanced tasks. We develop novel momentum methods with a better iteration of $O (1/epsilon4)$ for finding an $epsilon$stationary solution. We also design a novel family of adaptive methods with the same complexity of $O (1/epsilon4)$, which enjoy faster convergence in practice.
  arXiv  Detail & Related papers  (2021-07-02T16:21:52Z)
• Robust Value Iteration for Continuous Control Tasks [99.00362538261972]
  When transferring a control policy from simulation to a physical system, the policy needs to be robust to variations in the dynamics to perform well. We present Robust Fitted Value Iteration, which uses dynamic programming to compute the optimal value function on the compact state domain. We show that robust value is more robust compared to deep reinforcement learning algorithm and the non-robust version of the algorithm.
  arXiv  Detail & Related papers  (2021-05-25T19:48:35Z)
• Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence [66.83161885378192]
  Area under ROC (AUROC) and precision-recall curves (AUPRC) are common metrics for evaluating classification performance for imbalanced problems. We propose a technical method to optimize AUPRC for deep learning.
  arXiv  Detail & Related papers  (2021-04-18T06:22:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.