ConTinTin: Continual Learning from Task Instructions
- URL: http://arxiv.org/abs/2203.08512v1
- Date: Wed, 16 Mar 2022 10:27:18 GMT
- Title: ConTinTin: Continual Learning from Task Instructions
- Authors: Wenpeng Yin, Jia Li, Caiming Xiong
- Abstract summary: This work defines a new learning paradigm ConTinTin, in which a system should learn a sequence of new tasks one by one, each task is explained by a piece of textual instruction.
To our knowledge, this is the first time to study ConTinTin in NLP.
- Score: 101.36836925135091
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The mainstream machine learning paradigms for NLP often work with two
underlying presumptions. First, the target task is predefined and static, a
system just needs to learn to solve it exclusively. Second, the supervision of
a task mainly comes from a set of labeled examples. A question arises: how to
build a system that can keep learning new tasks from their instructions? This
work defines a new learning paradigm ConTinTin (Continual Learning from Task
Instructions), in which a system should learn a sequence of new tasks one by
one, each task is explained by a piece of textual instruction. The system is
required to (i) generate the expected outputs of a new task by learning from
its instruction, (ii) transfer the knowledge acquired from upstream tasks to
help solve downstream tasks (i.e, forward-transfer), and (iii) retain or even
improve the performance on earlier tasks after learning new tasks (i.e.,
backward-transfer). This new problem is studied on a stream of more than 60
tasks, each equipped with an instruction. Technically, our method
InstructionSpeak contains two strategies that make full use of task
instructions to improve forward-transfer and backward-transfer: one is to learn
from the negative output, the other is to re-visit instructions of prior tasks.
To our knowledge, this is the first time to study ConTinTin in NLP. In addition
to the problem formulation and our promising approach, this work also
contributes to providing rich analyses for the community to better understand
this novel learning problem.
Related papers
- Continual Deep Reinforcement Learning with Task-Agnostic Policy Distillation [0.0]
The Task-Agnostic Policy Distillation (TAPD) framework is introduced.
This paper addresses the problem of continual learning.
By utilizing task-agnostic distilled knowledge, the agent can solve downstream tasks more efficiently.
arXiv Detail & Related papers (2024-11-25T16:18:39Z) - Large Language Model Instruction Following: A Survey of Progresses and Challenges [15.94137745420097]
This paper tries to summarize and provide insights to the current research on instruction following.
To our knowledge, this is the first comprehensive survey about instruction following.
arXiv Detail & Related papers (2023-03-18T19:17:47Z) - Robustness of Learning from Task Instructions [15.462970803323563]
Traditional supervised learning mostly works on individual tasks and requires training on a large set of task-specific examples.
To build a system that can quickly and easily generalize to new tasks, task instructions have been adopted as an emerging trend of supervision.
This work investigates the system robustness when the instructions of new tasks are (i) manipulated, (ii) paraphrased, or (iii) from different levels of conciseness.
arXiv Detail & Related papers (2022-12-07T17:54:59Z) - Task Compass: Scaling Multi-task Pre-training with Task Prefix [122.49242976184617]
Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks.
We propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks.
Our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships.
arXiv Detail & Related papers (2022-10-12T15:02:04Z) - Transferring Knowledge for Reinforcement Learning in Contact-Rich
Manipulation [10.219833196479142]
We address the challenge of transferring knowledge within a family of similar tasks by leveraging multiple skill priors.
Our method learns a latent action space representing the skill embedding from demonstrated trajectories for each prior task.
We have evaluated our method on a set of peg-in-hole insertion tasks and demonstrate better generalization to new tasks that have never been encountered during training.
arXiv Detail & Related papers (2022-09-19T10:31:13Z) - Fast Inference and Transfer of Compositional Task Structures for
Few-shot Task Generalization [101.72755769194677]
We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph.
Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks.
Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
arXiv Detail & Related papers (2022-05-25T10:44:25Z) - Continual Prompt Tuning for Dialog State Tracking [58.66412648276873]
A desirable dialog system should be able to continually learn new skills without forgetting old ones.
We present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks.
arXiv Detail & Related papers (2022-03-13T13:22:41Z) - Parrot: Data-Driven Behavioral Priors for Reinforcement Learning [79.32403825036792]
We propose a method for pre-training behavioral priors that can capture complex input-output relationships observed in successful trials.
We show how this learned prior can be used for rapidly learning new tasks without impeding the RL agent's ability to try out novel behaviors.
arXiv Detail & Related papers (2020-11-19T18:47:40Z) - Measuring and Harnessing Transference in Multi-Task Learning [58.48659733262734]
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
We analyze the dynamics of information transfer, or transference, across tasks throughout training.
arXiv Detail & Related papers (2020-10-29T08:25:43Z) - Representation Ensembling for Synergistic Lifelong Learning with
Quasilinear Complexity [17.858926093389737]
In lifelong learning, data are used to improve performance not only on the current task, but also on previously encountered, and as yet unencountered tasks.
Our key insight is that we can synergistically ensemble representations -- that were learned independently on disparate tasks -- to enable both forward and backward transfer.
arXiv Detail & Related papers (2020-04-27T16:16:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.