Covariance Matrix Adaptation Evolution Strategy without a matrix

• URL: http://arxiv.org/abs/2601.00102v1
• Date: Wed, 31 Dec 2025 20:35:58 GMT
• Title: Covariance Matrix Adaptation Evolution Strategy without a matrix
• Abstract summary: We introduce a version of CMA-ES that uses no covariance matrix at all.<n>We prove that the probability distribution of individuals generated by the proposed method is identical to that of the standard CMA-ES.<n>The matrix-free CMA-ES converges faster than the matrix-based, and usually yields the results of a comparable or superior quality.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a highly effective optimization technique. A primary challenge when applying CMA-ES in high dimensionality is sampling from a multivariate normal distribution with an arbitrary covariance matrix, which involves its decomposition. The cubic complexity of this process is the main obstacle to applying CMA-ES in highdimensional spaces. We introduce a version of CMA-ES that uses no covariance matrix at all. In the proposed matrix-free CMA-ES, an archive stores the vectors of differences between individuals and the midpoint, normalized by the step size. New individuals are generated as the weighted combinations of the vectors from the archive. We prove that the probability distribution of individuals generated by the proposed method is identical to that of the standard CMA-ES. Experimental results show that reducing the archive size to store only a fixed number of the most recent populations is sufficient, without compromising optimization efficiency. The matrix-free and matrix-based CMA-ES achieve comparable results on the quadratic function when the step-size adaptation is turned off. When coupled with the step-size adaptation method, the matrix-free CMA-ES converges faster than the matrix-based, and usually yields the results of a comparable or superior quality, according to the results obtained for the CEC'2017 benchmark suite. Presented approach simplifies the algorithm, offers a novel perspective on covariance matrix adaptation, and serves as a stepping stone toward even more efficient methods.

Related papers

• SOLVAR: Fast covariance-based heterogeneity analysis with pose refinement for cryo-EM [1.739627424017212]
  Cryo-electron microscopy (cryo-EM) has emerged as a powerful technique for resolving the three-dimensional structures of macromolecules.<n>A key challenge in cryo-EM is characterizing continuous heterogeneity, where molecules adopt a continuum of conformational states.<n>Covariance-based methods offer a principled approach to modeling structural variability.
  arXiv  Detail & Related papers  (2026-02-19T18:28:46Z)
• Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation [53.88562288388169]
  A common strategy for. Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViTs) involves adapting the model to downstream tasks. We propose a novel PEFT approach inspired by Singular Value Decomposition (SVD) for representing the adaptation matrix. SVD decomposes a matrix into the product of a left unitary matrix, a diagonal matrix of scaling values, and a right unitary matrix.
  arXiv  Detail & Related papers  (2024-10-30T12:08:30Z)
• CatCMA : Stochastic Optimization for Mixed-Category Problems [9.13074910982872]
  Black-box optimization problems often require simultaneously optimizing different types of variables, such as continuous, integer, and categorical variables.<n>Several Bayesian optimization methods can deal with mixed-category black-box optimization (MC-BBO), but they suffer from a lack of scalability to high-dimensional problems and internal computational cost.<n>This paper proposes CatCMA, a new optimization method for MC-BBO problems.
  arXiv  Detail & Related papers  (2024-05-16T10:11:18Z)
• An Efficient Algorithm for Clustered Multi-Task Compressive Sensing [60.70532293880842]
  Clustered multi-task compressive sensing is a hierarchical model that solves multiple compressive sensing tasks. The existing inference algorithm for this model is computationally expensive and does not scale well in high dimensions. We propose a new algorithm that substantially accelerates model inference by avoiding the need to explicitly compute these covariance matrices.
  arXiv  Detail & Related papers  (2023-09-30T15:57:14Z)
• The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions [32.30527731746912]
  We propose a novel approximate filtering and smoothing method which propagates lowrank approximations of low-rank matrices. Our method reduces computational complexity from cubic (for the Kalman filter) to emphquadratic in the state-space size in the worst-case.
  arXiv  Detail & Related papers  (2023-06-13T13:50:31Z)
• Improved Privacy-Preserving PCA Using Optimized Homomorphic Matrix Multiplication [0.0]
  Principal Component Analysis (PCA) is a pivotal technique widely utilized in the realms of machine learning and data analysis. In recent years, there have been endeavors to utilize homomorphic encryption in privacy-preserving PCA algorithms for the secure cloud computing scenario. We propose a novel approach to privacy-preserving PCA that addresses these limitations, resulting in superior efficiency, accuracy, and scalability compared to previous approaches.
  arXiv  Detail & Related papers  (2023-05-27T02:51:20Z)
• Classification of BCI-EEG based on augmented covariance matrix [0.0]
  We propose a new framework based on the augmented covariance extracted from an autoregressive model to improve motor imagery classification. We will test our approach on several datasets and several subjects using the MOABB framework.
  arXiv  Detail & Related papers  (2023-02-09T09:04:25Z)
• Asymmetric Scalable Cross-modal Hashing [51.309905690367835]
  Cross-modal hashing is a successful method to solve large-scale multimedia retrieval issue. We propose a novel Asymmetric Scalable Cross-Modal Hashing (ASCMH) to address these issues. Our ASCMH outperforms the state-of-the-art cross-modal hashing methods in terms of accuracy and efficiency.
  arXiv  Detail & Related papers  (2022-07-26T04:38:47Z)
• Covariance Matrix Adaptation MAP-Annealing [5.5216935362914485]
  We propose a new quality diversity algorithm, Covariance Matrix Adaptation MAP-Annealing (CMA-MAE) We provide theoretical justifications for the new algorithm with respect to each limitation. Our theory informs our experiments, which show that CMA-MAE achieves state-of-the-art performance and robustness.
  arXiv  Detail & Related papers  (2022-05-22T06:01:02Z)
• Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection [138.97647716793333]
  We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization. We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
  arXiv  Detail & Related papers  (2020-12-29T04:08:38Z)
• Effective Dimension Adaptive Sketching Methods for Faster Regularized Least-Squares Optimization [56.05635751529922]
  We propose a new randomized algorithm for solving L2-regularized least-squares problems based on sketching. We consider two of the most popular random embeddings, namely, Gaussian embeddings and the Subsampled Randomized Hadamard Transform (SRHT)
  arXiv  Detail & Related papers  (2020-06-10T15:00:09Z)
• Multi-Objective Matrix Normalization for Fine-grained Visual Recognition [153.49014114484424]
  Bilinear pooling achieves great success in fine-grained visual recognition (FGVC) Recent methods have shown that the matrix power normalization can stabilize the second-order information in bilinear features. We propose an efficient Multi-Objective Matrix Normalization (MOMN) method that can simultaneously normalize a bilinear representation.
  arXiv  Detail & Related papers  (2020-03-30T08:40:35Z)
• The Hessian Estimation Evolution Strategy [3.756550107432323]
  We present a novel black box optimization algorithm called Hessian Estimation Evolution Strategy. The algorithm updates the covariance matrix of its sampling distribution by directly estimating the curvature of the objective function.
  arXiv  Detail & Related papers  (2020-03-30T08:01:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.