On Parameter Tuning in Meta-learning for Computer Vision

• URL: http://arxiv.org/abs/2003.00837v1
• Date: Tue, 11 Feb 2020 15:07:30 GMT
• Title: On Parameter Tuning in Meta-learning for Computer Vision
• Authors: Farid Ghareh Mohammadi, M. Hadi Amini, and Hamid R. Arabnia
• Abstract summary: In this paper, we investigate mage recognition for unseen categories of a given dataset with limited training information. We deploy a zero-shot learning (ZSL) algorithm to achieve this goal. We also explore the effect of parameter tuning on performance of semantic auto-encoder (SAE)
• Score: 2.3513645401551333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning to learn plays a pivotal role in meta-learning (MTL) to obtain an optimal learning model. In this paper, we investigate mage recognition for unseen categories of a given dataset with limited training information. We deploy a zero-shot learning (ZSL) algorithm to achieve this goal. We also explore the effect of parameter tuning on performance of semantic auto-encoder (SAE). We further address the parameter tuning problem for meta-learning, especially focusing on zero-shot learning. By combining different embedded parameters, we improved the accuracy of tuned-SAE. Advantages and disadvantages of parameter tuning and its application in image classification are also explored.

Related papers

• Learn To Learn More Precisely [30.825058308218047]
  "Learn to learn more precisely" aims to make the model learn precise target knowledge from data. We propose a simple and effective meta-learning framework named Meta Self-Distillation(MSD) to maximize the consistency of learned knowledge. MSD exhibits remarkable performance in few-shot classification tasks in both standard and augmented scenarios.
  arXiv  Detail & Related papers  (2024-08-08T17:01:26Z)
• Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters [65.15700861265432]
  We present a parameter-efficient continual learning framework to alleviate long-term forgetting in incremental learning with vision-language models. Our approach involves the dynamic expansion of a pre-trained CLIP model, through the integration of Mixture-of-Experts (MoE) adapters. To preserve the zero-shot recognition capability of vision-language models, we introduce a Distribution Discriminative Auto-Selector.
  arXiv  Detail & Related papers  (2024-03-18T08:00:23Z)
• Unsupervised Representation Learning to Aid Semi-Supervised Meta Learning [16.534014215010757]
  We propose a one-shot unsupervised meta-learning to learn latent representation of training samples. A temperature-scaled cross-entropy loss is used in the inner loop of meta-learning to prevent overfitting. The proposed method is model agnostic and can aid any meta-learning model to improve accuracy.
  arXiv  Detail & Related papers  (2023-10-19T18:25:22Z)
• Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective [106.92016199403042]
  We empirically investigate knowledge transfer from larger to smaller models through a parametric perspective. We employ sensitivity-based techniques to extract and align knowledge-specific parameters between different large language models. Our findings highlight the critical factors contributing to the process of parametric knowledge transfer.
  arXiv  Detail & Related papers  (2023-10-17T17:58:34Z)
• Retrieval-Augmented Meta Learning for Low-Resource Text Classification [22.653220906899612]
  We propose a meta-learning based method called Retrieval-Augmented Meta Learning(RAML) It uses parameterization for inference but also retrieves non-parametric knowledge from an external corpus to make inferences. RAML significantly outperforms current SOTA low-resource text classification models.
  arXiv  Detail & Related papers  (2023-09-10T10:05:03Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• Dynamic Regret Analysis for Online Meta-Learning [0.0]
  The online meta-learning framework has arisen as a powerful tool for the continual lifelong learning setting. This formulation involves two levels: outer level which learns meta-learners and inner level which learns task-specific models. We establish performance in terms of dynamic regret which handles changing environments from a global prospective. We carry out our analyses in a setting, and in expectation prove a logarithmic local dynamic regret which explicitly depends on the total number of iterations.
  arXiv  Detail & Related papers  (2021-09-29T12:12:59Z)
• Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks [62.63852372239708]
  Meta learning aims to learn the learning methodology for machine learning from observed tasks, so as to generalize to new query tasks. We interpret such learning methodology as learning an explicit hyper- parameter prediction function shared by all training tasks. Such setting guarantees that the meta-learned learning methodology is able to flexibly fit diverse query tasks.
  arXiv  Detail & Related papers  (2021-07-06T04:05:08Z)
• Continual Learning via Bit-Level Information Preserving [88.32450740325005]
  We study the continual learning process through the lens of information theory. We propose Bit-Level Information Preserving (BLIP) that preserves the information gain on model parameters. BLIP achieves close to zero forgetting while only requiring constant memory overheads throughout continual learning.
  arXiv  Detail & Related papers  (2021-05-10T15:09:01Z)
• One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video [71.43912903508765]
  MDAL is a dynamic online adaptive learning scheme for instrument segmentation in robot-assisted surgery. It learns the general knowledge of instruments and the fast adaptation ability through the video-specific meta-learning paradigm. It outperforms other state-of-the-art methods on two datasets.
  arXiv  Detail & Related papers  (2021-03-24T05:02:18Z)
• A Primal-Dual Subgradient Approachfor Fair Meta Learning [23.65344558042896]
  Few shot meta-learning is well-known with its fast-adapted capability and accuracy generalization onto unseen tasks. We propose a Primal-Dual Fair Meta-learning framework, namely PDFM, which learns to train fair machine learning models using only a few examples.
  arXiv  Detail & Related papers  (2020-09-26T19:47:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.