Interval Bound Interpolation for Few-shot Learning with Few Tasks
- URL: http://arxiv.org/abs/2204.03511v4
- Date: Sun, 7 May 2023 18:15:26 GMT
- Title: Interval Bound Interpolation for Few-shot Learning with Few Tasks
- Authors: Shounak Datta, Sankha Subhra Mullick, Anish Chakrabarty, Swagatam Das
- Abstract summary: Few-shot learning aims to transfer the knowledge acquired from training on a diverse set of tasks to unseen tasks with a limited amount of labeled data.
We introduce the notion of interval bounds from the provably robust training literature to few-shot learning.
We then use a novel strategy to artificially form new tasks for training by interpolating between the available tasks and their respective interval bounds.
- Score: 15.85259386116784
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Few-shot learning aims to transfer the knowledge acquired from training on a
diverse set of tasks to unseen tasks from the same task distribution with a
limited amount of labeled data. The underlying requirement for effective
few-shot generalization is to learn a good representation of the task manifold.
This becomes more difficult when only a limited number of tasks are available
for training. In such a few-task few-shot setting, it is beneficial to
explicitly preserve the local neighborhoods from the task manifold and exploit
this to generate artificial tasks for training. To this end, we introduce the
notion of interval bounds from the provably robust training literature to
few-shot learning. The interval bounds are used to characterize neighborhoods
around the training tasks. These neighborhoods can then be preserved by
minimizing the distance between a task and its respective bounds. We then use a
novel strategy to artificially form new tasks for training by interpolating
between the available tasks and their respective interval bounds. We apply our
framework to both model-agnostic meta-learning as well as prototype-based
metric-learning paradigms. The efficacy of our proposed approach is evident
from the improved performance on several datasets from diverse domains compared
to current methods.
Related papers
- $α$VIL: Learning to Leverage Auxiliary Tasks for Multitask Learning [3.809702129519642]
Multitask Learning aims to train a range of (usually related) tasks with the help of a shared model.
It becomes important to estimate the positive or negative influence auxiliary tasks will have on the target.
We propose a novel method called $alpha$Variable Learning ($alpha$VIL) that is able to adjust task weights dynamically during model training.
arXiv Detail & Related papers (2024-05-13T14:12:33Z) - Data-CUBE: Data Curriculum for Instruction-based Sentence Representation
Learning [85.66907881270785]
We propose a data curriculum method, namely Data-CUBE, that arranges the orders of all the multi-task data for training.
In the task level, we aim to find the optimal task order to minimize the total cross-task interference risk.
In the instance level, we measure the difficulty of all instances per task, then divide them into the easy-to-difficult mini-batches for training.
arXiv Detail & Related papers (2024-01-07T18:12:20Z) - Distribution Matching for Multi-Task Learning of Classification Tasks: a
Large-Scale Study on Faces & Beyond [62.406687088097605]
Multi-Task Learning (MTL) is a framework, where multiple related tasks are learned jointly and benefit from a shared representation space.
We show that MTL can be successful with classification tasks with little, or non-overlapping annotations.
We propose a novel approach, where knowledge exchange is enabled between the tasks via distribution matching.
arXiv Detail & Related papers (2024-01-02T14:18:11Z) - Multitask Learning with No Regret: from Improved Confidence Bounds to
Active Learning [79.07658065326592]
Quantifying uncertainty in the estimated tasks is of pivotal importance for many downstream applications, such as online or active learning.
We provide novel multitask confidence intervals in the challenging setting when neither the similarity between tasks nor the tasks' features are available to the learner.
We propose a novel online learning algorithm that achieves such improved regret without knowing this parameter in advance.
arXiv Detail & Related papers (2023-08-03T13:08:09Z) - Derivative Free Weight-space Ensembling [0.0]
We introduce Derivative Free Weight-space Ensembling (DFWE), a new few-sample task transfer approach for open-domain dialogue.
We finetune each of the expert models on the target task, approaching the target task from several distinct knowledge bases.
We linearly interpolate between the model weights using a gradient-free-optimization algorithm, to efficiently find a good weighting.
arXiv Detail & Related papers (2023-07-07T10:42:44Z) - ImpressLearn: Continual Learning via Combined Task Impressions [0.0]
This work proposes a new method to sequentially train a deep neural network on multiple tasks without suffering catastrophic forgetting.
We show that simply learning a linear combination of a small number of task-specific masks on a randomly backbone network is sufficient to both retain accuracy on previously learned tasks, as well as achieve high accuracy on new tasks.
arXiv Detail & Related papers (2022-10-05T02:28:25Z) - Set-based Meta-Interpolation for Few-Task Meta-Learning [79.4236527774689]
We propose a novel domain-agnostic task augmentation method, Meta-Interpolation, to densify the meta-training task distribution.
We empirically validate the efficacy of Meta-Interpolation on eight datasets spanning across various domains.
arXiv Detail & Related papers (2022-05-20T06:53:03Z) - Learning Multiple Dense Prediction Tasks from Partially Annotated Data [41.821234589075445]
We look at jointly learning of multiple dense prediction tasks on partially annotated data, which we call multi-task partially-supervised learning.
We propose a multi-task training procedure that successfully leverages task relations to supervise its multi-task learning when data is partially annotated.
We rigorously demonstrate that our proposed method effectively exploits the images with unlabelled tasks and outperforms existing semi-supervised learning approaches and related methods on three standard benchmarks.
arXiv Detail & Related papers (2021-11-29T19:03:12Z) - Conditional Meta-Learning of Linear Representations [57.90025697492041]
Standard meta-learning for representation learning aims to find a common representation to be shared across multiple tasks.
In this work we overcome this issue by inferring a conditioning function, mapping the tasks' side information into a representation tailored to the task at hand.
We propose a meta-algorithm capable of leveraging this advantage in practice.
arXiv Detail & Related papers (2021-03-30T12:02:14Z) - Adversarial Continual Learning [99.56738010842301]
We propose a hybrid continual learning framework that learns a disjoint representation for task-invariant and task-specific features.
Our model combines architecture growth to prevent forgetting of task-specific skills and an experience replay approach to preserve shared skills.
arXiv Detail & Related papers (2020-03-21T02:08:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.