Vectorial Genetic Programming -- Optimizing Segments for Feature
Extraction
- URL: http://arxiv.org/abs/2303.03200v1
- Date: Fri, 3 Mar 2023 10:08:10 GMT
- Title: Vectorial Genetic Programming -- Optimizing Segments for Feature
Extraction
- Authors: Philipp Fleck, Stephan Winkler, Michael Kommenda, Michael Affenzeller
- Abstract summary: Vec-GP allows aggregating vectors only over a limited segment of the vector instead of the whole vector.
This paper formalizes an optimization problem to analyze different strategies for optimizing a window for aggregation functions.
- Score: 2.561649173827544
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Vectorial Genetic Programming (Vec-GP) extends GP by allowing vectors as
input features along regular, scalar features, using them by applying
arithmetic operations component-wise or aggregating vectors into scalars by
some aggregation function. Vec-GP also allows aggregating vectors only over a
limited segment of the vector instead of the whole vector, which offers great
potential but also introduces new parameters that GP has to optimize. This
paper formalizes an optimization problem to analyze different strategies for
optimizing a window for aggregation functions. Different strategies are
presented, included random and guided sampling, where the latter leverages
information from an approximated gradient. Those strategies can be applied as a
simple optimization algorithm, which itself ca be applied inside a specialized
mutation operator within GP. The presented results indicate, that the different
random sampling strategies do not impact the overall algorithm performance
significantly, and that the guided strategies suffer from becoming stuck in
local optima. However, results also indicate, that there is still potential in
discovering more efficient algorithms that could outperform the presented
strategies.
Related papers
- A Principle for Global Optimization with Gradients [0.0]
This work demonstrates the utility of gradients for the global optimization of certain differentiable functions with many suboptimal local minima.
Experiments measure the quality of non-local search directions as well as the performance of a proposed simplistic algorithm.
arXiv Detail & Related papers (2023-08-18T13:39:29Z) - Thompson sampling for improved exploration in GFlowNets [75.89693358516944]
Generative flow networks (GFlowNets) are amortized variational inference algorithms that treat sampling from a distribution over compositional objects as a sequential decision-making problem with a learnable action policy.
We show in two domains that TS-GFN yields improved exploration and thus faster convergence to the target distribution than the off-policy exploration strategies used in past work.
arXiv Detail & Related papers (2023-06-30T14:19:44Z) - An Empirical Evaluation of Zeroth-Order Optimization Methods on
AI-driven Molecule Optimization [78.36413169647408]
We study the effectiveness of various ZO optimization methods for optimizing molecular objectives.
We show the advantages of ZO sign-based gradient descent (ZO-signGD)
We demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite.
arXiv Detail & Related papers (2022-10-27T01:58:10Z) - SADAM: Stochastic Adam, A Stochastic Operator for First-Order
Gradient-based Optimizer [8.93274096260726]
We propose, analyze, and generalize a strategy performed as an operator for a first-order descent algorithm.
Unlike existing algorithms, the proposed strategy does not require any batches and sampling techniques.
arXiv Detail & Related papers (2022-05-20T15:20:19Z) - Bayesian Optimization for auto-tuning GPU kernels [0.0]
Finding optimal parameter configurations for GPU kernels is a non-trivial exercise for large search spaces, even when automated.
We introduce a novel contextual exploration factor as well as new acquisition functions with improved scalability, combined with an informed function selection mechanism.
arXiv Detail & Related papers (2021-11-26T11:26:26Z) - Bregman Gradient Policy Optimization [97.73041344738117]
We design a Bregman gradient policy optimization for reinforcement learning based on Bregman divergences and momentum techniques.
VR-BGPO reaches the best complexity $tilde(epsilon-3)$ for finding an $epsilon$stationary point only requiring one trajectory at each iteration.
arXiv Detail & Related papers (2021-06-23T01:08:54Z) - Optimistic search strategy: Change point detection for large-scale data
via adaptive logarithmic queries [1.3212010735248336]
Change point detection is often formulated as a search for the maximum of a gain function describing improved fits when segmenting the data.
We propose optimistic search strategies with $O(log T)$ exploiting specific structure of the gain function.
arXiv Detail & Related papers (2020-10-20T11:09:52Z) - Global Optimization of Gaussian processes [52.77024349608834]
We propose a reduced-space formulation with trained Gaussian processes trained on few data points.
The approach also leads to significantly smaller and computationally cheaper sub solver for lower bounding.
In total, we reduce time convergence by orders of orders of the proposed method.
arXiv Detail & Related papers (2020-05-21T20:59:11Z) - Optimization of Graph Total Variation via Active-Set-based Combinatorial
Reconditioning [48.42916680063503]
We propose a novel adaptive preconditioning strategy for proximal algorithms on this problem class.
We show that nested-forest decomposition of the inactive edges yields a guaranteed local linear convergence rate.
Our results suggest that local convergence analysis can serve as a guideline for selecting variable metrics in proximal algorithms.
arXiv Detail & Related papers (2020-02-27T16:33:09Z) - Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization [71.03797261151605]
Adaptivity is an important yet under-studied property in modern optimization theory.
Our algorithm is proved to achieve the best-available convergence for non-PL objectives simultaneously while outperforming existing algorithms for PL objectives.
arXiv Detail & Related papers (2020-02-13T05:42:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.