Meta-Learning via Classifier(-free) Guidance
- URL: http://arxiv.org/abs/2210.08942v1
- Date: Mon, 17 Oct 2022 11:09:35 GMT
- Title: Meta-Learning via Classifier(-free) Guidance
- Authors: Elvis Nava, Seijin Kobayashi, Yifei Yin, Robert K. Katzschmann,
Benjamin F. Grewe
- Abstract summary: State-of-the-art meta-learning techniques do not optimize for zero-shot adaptation to unseen tasks.
We propose meta-learning techniques that use natural language guidance to achieve higher zero-shot performance.
- Score: 5.812784742024491
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: State-of-the-art meta-learning techniques do not optimize for zero-shot
adaptation to unseen tasks, a setting in which humans excel. On the contrary,
meta-learning algorithms learn hyperparameters and weight initializations that
explicitly optimize for few-shot learning performance. In this work, we take
inspiration from recent advances in generative modeling and
language-conditioned image synthesis to propose meta-learning techniques that
use natural language guidance to achieve higher zero-shot performance compared
to the state-of-the-art. We do so by recasting the meta-learning problem as a
multi-modal generative modeling problem: given a task, we consider its adapted
neural network weights and its natural language description as equivalent
multi-modal task representations. We first train an unconditional generative
hypernetwork model to produce neural network weights; then we train a second
"guidance" model that, given a natural language task description, traverses the
hypernetwork latent space to find high-performance task-adapted weights in a
zero-shot manner. We explore two alternative approaches for latent space
guidance: "HyperCLIP"-based classifier guidance and a conditional Hypernetwork
Latent Diffusion Model ("HyperLDM"), which we show to benefit from the
classifier-free guidance technique common in image generation. Finally, we
demonstrate that our approaches outperform existing meta-learning methods with
zero-shot learning experiments on our Meta-VQA dataset, which we specifically
constructed to reflect the multi-modal meta-learning setting.
Related papers
- Unified Generative and Discriminative Training for Multi-modal Large Language Models [88.84491005030316]
Generative training has enabled Vision-Language Models (VLMs) to tackle various complex tasks.
Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval.
This paper proposes a unified approach that integrates the strengths of both paradigms.
arXiv Detail & Related papers (2024-11-01T01:51:31Z) - ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning [49.447777286862994]
ConML is a universal meta-learning framework that can be applied to various meta-learning algorithms.
We demonstrate that ConML integrates seamlessly with optimization-based, metric-based, and amortization-based meta-learning algorithms.
arXiv Detail & Related papers (2024-10-08T12:22:10Z) - Meta-Task Prompting Elicits Embeddings from Large Language Models [54.757445048329735]
We introduce a new unsupervised text embedding method, Meta-Task Prompting with Explicit One-Word Limitation.
We generate high-quality sentence embeddings from Large Language Models without the need for model fine-tuning.
Our findings suggest a new scaling law, offering a versatile and resource-efficient approach for embedding generation across diverse scenarios.
arXiv Detail & Related papers (2024-02-28T16:35:52Z) - Diffusion-Based Neural Network Weights Generation [80.89706112736353]
D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning.
Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation.
Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
arXiv Detail & Related papers (2024-02-28T08:34:23Z) - Model-Based Reinforcement Learning with Multi-Task Offline Pretraining [59.82457030180094]
We present a model-based RL method that learns to transfer potentially useful dynamics and action demonstrations from offline data to a novel task.
The main idea is to use the world models not only as simulators for behavior learning but also as tools to measure the task relevance.
We demonstrate the advantages of our approach compared with the state-of-the-art methods in Meta-World and DeepMind Control Suite.
arXiv Detail & Related papers (2023-06-06T02:24:41Z) - Meta Learning to Bridge Vision and Language Models for Multimodal
Few-Shot Learning [38.37682598345653]
We introduce a multimodal meta-learning approach to bridge the gap between vision and language models.
We define a meta-mapper network, acting as a meta-learner, to efficiently bridge frozen large-scale vision and language models.
We evaluate our approach on recently proposed multimodal few-shot benchmarks, measuring how rapidly the model can bind novel visual concepts to words.
arXiv Detail & Related papers (2023-02-28T17:46:18Z) - Unsupervised Meta-Learning through Latent-Space Interpolation in
Generative Models [11.943374020641214]
We describe an approach that generates meta-tasks using generative models.
We find that the proposed approach, LAtent Space Interpolation Unsupervised Meta-learning (LASIUM), outperforms or is competitive with current unsupervised learning baselines.
arXiv Detail & Related papers (2020-06-18T02:10:56Z) - Pre-training Text Representations as Meta Learning [113.3361289756749]
We introduce a learning algorithm which directly optimize model's ability to learn text representations for effective learning of downstream tasks.
We show that there is an intrinsic connection between multi-task pre-training and model-agnostic meta-learning with a sequence of meta-train steps.
arXiv Detail & Related papers (2020-04-12T09:05:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.