Learning Abstract Task Representations

• URL: http://arxiv.org/abs/2101.07852v3
• Date: Thu, 28 Jan 2021 20:07:20 GMT
• Title: Learning Abstract Task Representations
• Authors: Mikhail M. Meskhi, Adriano Rivolli, Rafael G. Mantovani, Ricardo Vilalta
• Abstract summary: We propose a method to induce new abstract meta-features as latent variables in a deep neural network. We demonstrate our methodology using a deep neural network as a feature extractor.
• Score: 0.6690874707758511
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A proper form of data characterization can guide the process of learning-algorithm selection and model-performance estimation. The field of meta-learning has provided a rich body of work describing effective forms of data characterization using different families of meta-features (statistical, model-based, information-theoretic, topological, etc.). In this paper, we start with the abundant set of existing meta-features and propose a method to induce new abstract meta-features as latent variables in a deep neural network. We discuss the pitfalls of using traditional meta-features directly and argue for the importance of learning high-level task properties. We demonstrate our methodology using a deep neural network as a feature extractor. We demonstrate that 1) induced meta-models mapping abstract meta-features to generalization performance outperform other methods by ~18% on average, and 2) abstract meta-features attain high feature-relevance scores.

Related papers

• Dynamics of Meta-learning Representation in the Teacher-student Scenario [8.099691748821114]
  Gradient-based meta-learning algorithms have gained popularity for their ability to train models on new tasks using limited data. We investigate the meta-learning dynamics of the non-linear two-layer neural networks trained on streaming tasks in the teach-student scenario.
  arXiv  Detail & Related papers  (2024-08-22T16:59:32Z)
• Automatic learning algorithm selection for classification via convolutional neural networks [0.0]
  The goal of this study is to learn the inherent structure of the data without identifying meta-features. Experiments with simulated datasets show that the proposed approach achieves nearly perfect performance in identifying linear and nonlinear patterns.
  arXiv  Detail & Related papers  (2023-05-16T01:57:01Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• MARS: Meta-Learning as Score Matching in the Function Space [79.73213540203389]
  We present a novel approach to extracting inductive biases from a set of related datasets. We use functional Bayesian neural network inference, which views the prior as a process and performs inference in the function space. Our approach can seamlessly acquire and represent complex prior knowledge by metalearning the score function of the data-generating process.
  arXiv  Detail & Related papers  (2022-10-24T15:14:26Z)
• Meta-Learning with Self-Improving Momentum Target [72.98879709228981]
  We propose Self-improving Momentum Target (SiMT) to improve the performance of a meta-learner. SiMT generates the target model by adapting from the temporal ensemble of the meta-learner. We show that SiMT brings a significant performance gain when combined with a wide range of meta-learning methods.
  arXiv  Detail & Related papers  (2022-10-11T06:45:15Z)
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach [80.8446673089281]
  We propose a new learning paradigm with graph representation and learning. Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
  arXiv  Detail & Related papers  (2021-10-09T09:02:45Z)
• Meta-Learning with Fewer Tasks through Task Interpolation [67.03769747726666]
  Current meta-learning algorithms require a large number of meta-training tasks, which may not be accessible in real-world scenarios. By meta-learning with task gradient (MLTI), our approach effectively generates additional tasks by randomly sampling a pair of tasks and interpolating the corresponding features and labels. Empirically, in our experiments on eight datasets from diverse domains, we find that the proposed general MLTI framework is compatible with representative meta-learning algorithms and consistently outperforms other state-of-the-art strategies.
  arXiv  Detail & Related papers  (2021-06-04T20:15:34Z)
• Predicting Scores of Medical Imaging Segmentation Methods with Meta-Learning [0.30458514384586394]
  We investigate meta-learning for segmentation across ten datasets of different organs and modalities. We use support vector regression and deep neural networks to learn the relationship between the meta-features and prior model performance. These results demonstrate the potential of meta-learning in medical imaging.
  arXiv  Detail & Related papers  (2020-05-08T07:47:52Z)
• Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks [55.66438591090072]
  We develop a better understanding of the underlying mechanics of meta-learning and the difference between models trained using meta-learning and models trained classically. We develop a regularizer which boosts the performance of standard training routines for few-shot classification.
  arXiv  Detail & Related papers  (2020-02-17T03:18:45Z)
• Towards explainable meta-learning [5.802346990263708]
  Meta-learning aims at discovering how different machine learning algorithms perform on a wide range of predictive tasks. State of the art approaches are focused on searching for the best meta-model but do not explain how these different aspects contribute to its performance. We propose techniques developed for eXplainable Artificial Intelligence (XAI) to examine and extract knowledge from black-box surrogate models.
  arXiv  Detail & Related papers  (2020-02-11T09:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.