DoE2Vec: Deep-learning Based Features for Exploratory Landscape Analysis

• URL: http://arxiv.org/abs/2304.01219v1
• Date: Fri, 31 Mar 2023 09:38:44 GMT
• Title: DoE2Vec: Deep-learning Based Features for Exploratory Landscape Analysis
• Authors: Bas van Stein, Fu Xing Long, Moritz Frenzel, Peter Krause, Markus Gitterle, Thomas B\"ack
• Abstract summary: We propose DoE2Vec, a variational autoencoder (VAE)-based methodology to learn optimization landscape characteristics. Unlike the classical exploratory landscape analysis (ELA) method, our approach does not require any feature engineering. For validation, we inspect the quality of latent reconstructions and analyze the latent representations using different experiments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose DoE2Vec, a variational autoencoder (VAE)-based methodology to learn optimization landscape characteristics for downstream meta-learning tasks, e.g., automated selection of optimization algorithms. Principally, using large training data sets generated with a random function generator, DoE2Vec self-learns an informative latent representation for any design of experiments (DoE). Unlike the classical exploratory landscape analysis (ELA) method, our approach does not require any feature engineering and is easily applicable for high dimensional search spaces. For validation, we inspect the quality of latent reconstructions and analyze the latent representations using different experiments. The latent representations not only show promising potentials in identifying similar (cheap-to-evaluate) surrogate functions, but also can significantly boost performances when being used complementary to the classical ELA features in classification tasks.

Related papers

• Parameter-Efficient and Memory-Efficient Tuning for Vision Transformer: A Disentangled Approach [87.8330887605381]
  We show how to adapt a pre-trained Vision Transformer to downstream recognition tasks with only a few learnable parameters. We synthesize a task-specific query with a learnable and lightweight module, which is independent of the pre-trained model. Our method achieves state-of-the-art performance under memory constraints, showcasing its applicability in real-world situations.
  arXiv  Detail & Related papers  (2024-07-09T15:45:04Z)
• Class-Specific Variational Auto-Encoder for Content-Based Image Retrieval [95.42181254494287]
  We propose a regularized loss for Variational Auto-Encoders (VAEs) forcing the model to focus on a given class of interest. As a result, the model learns to discriminate the data belonging to the class of interest from any other possibility. Experimental results show that the proposed method outperforms its competition in both in-domain and out-of-domain retrieval problems.
  arXiv  Detail & Related papers  (2023-04-23T19:51:25Z)
• Representation Learning with Multi-Step Inverse Kinematics: An Efficient and Optimal Approach to Rich-Observation RL [106.82295532402335]
  Existing reinforcement learning algorithms suffer from computational intractability, strong statistical assumptions, and suboptimal sample complexity. We provide the first computationally efficient algorithm that attains rate-optimal sample complexity with respect to the desired accuracy level. Our algorithm, MusIK, combines systematic exploration with representation learning based on multi-step inverse kinematics.
  arXiv  Detail & Related papers  (2023-04-12T14:51:47Z)
• Synthetic Sample Selection for Generalized Zero-Shot Learning [4.264192013842096]
  Generalized Zero-Shot Learning (GZSL) has emerged as a pivotal research domain in computer vision. This paper proposes a novel approach for synthetic feature selection using reinforcement learning.
  arXiv  Detail & Related papers  (2023-04-06T03:22:43Z)
• Self-Optimizing Feature Transformation [33.458785763961004]
  Feature transformation aims to extract a good representation (feature) space by mathematically transforming existing features. Current research focuses on domain knowledge-based feature engineering or learning latent representations. We present a self-optimizing framework for feature transformation.
  arXiv  Detail & Related papers  (2022-09-16T16:50:41Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• A Collection of Deep Learning-based Feature-Free Approaches for Characterizing Single-Objective Continuous Fitness Landscapes [0.0]
  Landscape insights are crucial for problem understanding as well as for assessing benchmark set diversity and composition. In this work we provide a collection of different approaches to characterize optimization landscapes. We demonstrate and validate our devised methods on the BBOB testbed and predict, with the help of Deep Learning.
  arXiv  Detail & Related papers  (2022-04-12T12:46:31Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)
• AEFE: Automatic Embedded Feature Engineering for Categorical Features [4.310748698480341]
  We propose an automatic feature engineering framework for representing categorical features, which consists of various components including custom paradigm feature construction and multiple feature selection. Experiments conducted on some typical e-commerce datasets indicate that our method outperforms the classical machine learning models and state-of-the-art deep learning models.
  arXiv  Detail & Related papers  (2021-10-19T07:22:59Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Towards Explainable Exploratory Landscape Analysis: Extreme Feature Selection for Classifying BBOB Functions [4.932130498861987]
  We show that a surprisingly small number of features -- often less than four -- can suffice to achieve a 98% accuracy. We show that the classification accuracy transfers to settings in which several instances are involved in training and testing.
  arXiv  Detail & Related papers  (2021-02-01T10:04:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.