Evaluating Meta-Feature Selection for the Algorithm Recommendation Problem

• URL: http://arxiv.org/abs/2106.03954v1
• Date: Mon, 7 Jun 2021 20:36:47 GMT
• Title: Evaluating Meta-Feature Selection for the Algorithm Recommendation Problem
• Authors: Geand Trindade Pereira, Moises Rocha dos Santos, Andre Carlos Ponce de Leon Ferreira de Carvalho
• Abstract summary: This paper presents an empirical analysis of Feature Selection and Feature Extraction in the meta-level for the Algorithm Recommendation problem. Dimensionality Reduction (DR) methods did not improve predictive performances in general. It is possible to obtain high predictive performance using around 20% of the original meta-features.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the popularity of Machine Learning (ML) solutions, algorithms and data have been released faster than the capacity of processing them. In this context, the problem of Algorithm Recommendation (AR) is receiving a significant deal of attention recently. This problem has been addressed in the literature as a learning task, often as a Meta-Learning problem where the aim is to recommend the best alternative for a specific dataset. For such, datasets encoded by meta-features are explored by ML algorithms that try to learn the mapping between meta-representations and the best technique to be used. One of the challenges for the successful use of ML is to define which features are the most valuable for a specific dataset since several meta-features can be used, which increases the meta-feature dimension. This paper presents an empirical analysis of Feature Selection and Feature Extraction in the meta-level for the AR problem. The present study was focused on three criteria: predictive performance, dimensionality reduction, and pipeline runtime. As we verified, applying Dimensionality Reduction (DR) methods did not improve predictive performances in general. However, DR solutions reduced about 80% of the meta-features, obtaining pretty much the same performance as the original setup but with lower runtimes. The only exception was PCA, which presented about the same runtime as the original meta-features. Experimental results also showed that various datasets have many non-informative meta-features and that it is possible to obtain high predictive performance using around 20% of the original meta-features. Therefore, due to their natural trend for high dimensionality, DR methods should be used for Meta-Feature Selection and Meta-Feature Extraction.

Related papers

• Learning to Learn with Indispensable Connections [6.040904021861969]
  We propose a novel meta-learning method called Meta-LTH that includes indispensible (necessary) connections. Our method improves the classification accuracy by approximately 2% (20-way 1-shot task setting) for omniglot dataset.
  arXiv  Detail & Related papers  (2023-04-06T04:53:13Z)
• Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction [97.99938802797377]
  Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in recommender systems. Recent deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. We propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper.
  arXiv  Detail & Related papers  (2022-06-28T03:28:15Z)
• Towards Meta-learned Algorithm Selection using Implicit Fidelity Information [13.750624267664156]
  IMFAS produces informative landmarks, easily enriched by arbitrary meta-features at a low computational cost. We show it is able to beat Successive Halving with at most half the fidelity sequence during test time.
  arXiv  Detail & Related papers  (2022-06-07T09:14:24Z)
• Does MAML Only Work via Feature Re-use? A Data Centric Perspective [19.556093984142418]
  We provide empirical results that shed some light on how meta-learned MAML representations function. We show that it is possible to define a family of synthetic benchmarks that result in a low degree of feature re-use. We conjecture the core challenge of re-thinking meta-learning is in the design of few-shot learning data sets and benchmarks.
  arXiv  Detail & Related papers  (2021-12-24T20:18:38Z)
• Meta-aprendizado para otimizacao de parametros de redes neurais [0.9558392439655014]
  We investigated the use of meta-learning to the optimization of ANNs. We performed a case study using meta-learning to choose the number of hidden nodes for networks.
  arXiv  Detail & Related papers  (2021-07-10T15:38:01Z)
• Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning [56.17603785248675]
  Model-agnostic meta-learning (MAML) has become a popular research area. Existing MAML algorithms rely on the episode' idea by sampling a few tasks and data points to update the meta-model at each iteration. This paper proposes memory-based algorithms for MAML that converge with vanishing error.
  arXiv  Detail & Related papers  (2021-06-09T08:47:58Z)
• Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification [59.326456778057384]
  We propose the Memory-based Multi-Source Meta-Learning framework to train a generalizable model for unseen domains. We also present a meta batch normalization layer (MetaBN) to diversify meta-test features. Experiments demonstrate that our M$3$L can effectively enhance the generalization ability of the model for unseen domains.
  arXiv  Detail & Related papers  (2020-12-01T11:38:16Z)
• Online Model Selection for Reinforcement Learning with Function Approximation [50.008542459050155]
  We present a meta-algorithm that adapts to the optimal complexity with $tildeO(L5/6 T2/3)$ regret. We also show that the meta-algorithm automatically admits significantly improved instance-dependent regret bounds.
  arXiv  Detail & Related papers  (2020-11-19T10:00:54Z)
• Meta-AAD: Active Anomaly Detection with Deep Reinforcement Learning [56.65934079419417]
  High false-positive rate is a long-standing challenge for anomaly detection algorithms. We propose Active Anomaly Detection with Meta-Policy (Meta-AAD), a novel framework that learns a meta-policy for query selection.
  arXiv  Detail & Related papers  (2020-09-16T01:47:42Z)
• Semi-Supervised Learning with Meta-Gradient [123.26748223837802]
  We propose a simple yet effective meta-learning algorithm in semi-supervised learning. We find that the proposed algorithm performs favorably against state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-08T08:48:56Z)
• Guarantees for Tuning the Step Size using a Learning-to-Learn Approach [18.838453594698166]
  We give meta-optimization guarantees for the learning-to-learn approach on a simple problem of tuning the step size for quadratic loss. Although there is a way to design the meta-objective so that the meta-gradient remains bounded, computing the meta-gradient directly using backpropagation leads to numerical issues. We also characterize when it is necessary to compute the meta-objective on a separate validation set to ensure the performance of the learned.
  arXiv  Detail & Related papers  (2020-06-30T02:59:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.