CITB: A Benchmark for Continual Instruction Tuning

• URL: http://arxiv.org/abs/2310.14510v1
• Date: Mon, 23 Oct 2023 02:42:32 GMT
• Title: CITB: A Benchmark for Continual Instruction Tuning
• Authors: Zihan Zhang, Meng Fang, Ling Chen, Mohammad-Reza Namazi-Rad
• Abstract summary: Continual learning (CL) is a paradigm that aims to replicate the human ability to learn and accumulate knowledge continually. Recent instruction tuning (IT) involves fine-tuning models to make them more adaptable to solving NLP tasks in general. We establish a benchmark consisting of learning and evaluation protocols to study various CL methods systematically.
• Score: 44.40322919392584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning (CL) is a paradigm that aims to replicate the human ability to learn and accumulate knowledge continually without forgetting previous knowledge and transferring it to new tasks. Recent instruction tuning (IT) involves fine-tuning models to make them more adaptable to solving NLP tasks in general. However, it is still uncertain how instruction tuning works in the context of CL tasks. This challenging yet practical problem is formulated as Continual Instruction Tuning (CIT). In this work, we establish a CIT benchmark consisting of learning and evaluation protocols. We curate two long dialogue task streams of different types, InstrDialog and InstrDialog++, to study various CL methods systematically. Our experiments show that existing CL methods do not effectively leverage the rich natural language instructions, and fine-tuning an instruction-tuned model sequentially can yield similar or better results. We further explore different aspects that might affect the learning of CIT. We hope this benchmark will facilitate more research in this direction.

Related papers

• SwitchCIT: Switching for Continual Instruction Tuning of Large Language Models [14.085371250265224]
  Large language models (LLMs) have exhibited impressive capabilities in various domains, particularly in general language understanding. However these models, trained on massive text data, may not be finely optimized for specific tasks triggered by instructions. This work addresses the catastrophic forgetting in continual instruction learning for LLMs through a switching mechanism for routing computations to parameter-efficient tuned models.
  arXiv  Detail & Related papers  (2024-07-16T14:37:33Z)
• InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions [29.682289142922752]
  InsCL dynamically replays previous data based on task similarity, calculated by Wasserstein Distance with instructions. InsCL achieves performance gains of 3.0 Relative Gain compared with Random Replay, and 27.96 Relative Gain compared with No Replay.
  arXiv  Detail & Related papers  (2024-03-18T03:10:36Z)
• SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models [71.78800549517298]
  Continual learning (CL) ability is vital for deploying large language models (LLMs) in the dynamic world. Existing methods devise the learning module to acquire task-specific knowledge with parameter-efficient tuning (PET) block and the selection module to pick out the corresponding one for the testing input. We propose a novel Shared Attention Framework (SAPT) to align the PET learning and selection via the Shared Attentive Learning & Selection module.
  arXiv  Detail & Related papers  (2024-01-16T11:45:03Z)
• POP: Prompt Of Prompts for Continual Learning [59.15888651733645]
  Continual learning (CL) aims to mimic the human ability to learn new concepts without catastrophic forgetting. We show that a foundation model equipped with POP learning is able to outperform classic CL methods by a significant margin.
  arXiv  Detail & Related papers  (2023-06-14T02:09:26Z)
• LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning [64.55001982176226]
  LIBERO is a novel benchmark of lifelong learning for robot manipulation. We focus on how to efficiently transfer declarative knowledge, procedural knowledge, or the mixture of both. We develop an extendible procedural generation pipeline that can in principle generate infinitely many tasks.
  arXiv  Detail & Related papers  (2023-06-05T23:32:26Z)
• Toward Sustainable Continual Learning: Detection and Knowledge Repurposing of Similar Tasks [31.095642850920385]
  We introduce a paradigm where the continual learner gets a sequence of mixed similar and dissimilar tasks. We propose a new continual learning framework that uses a task similarity detection function that does not require additional learning. Our experiments show that the proposed framework performs competitively on widely used computer vision benchmarks.
  arXiv  Detail & Related papers  (2022-10-11T19:35:30Z)
• Efficient Continual Learning with Modular Networks and Task-Driven Priors [31.03712334701338]
  Existing literature in Continual Learning (CL) has focused on overcoming catastrophic forgetting. We introduce a new modular architecture, whose modules represent atomic skills that can be composed to perform a certain task. Our learning algorithm leverages a task-driven prior over the exponential search space of all possible ways to combine modules, enabling efficient learning on long streams of tasks.
  arXiv  Detail & Related papers  (2020-12-23T12:42:16Z)
• Continual Lifelong Learning in Natural Language Processing: A Survey [3.9103337761169943]
  Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. It is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. We look at the problem of CL through the lens of various NLP tasks.
  arXiv  Detail & Related papers  (2020-12-17T18:44:36Z)
• Bilevel Continual Learning [76.50127663309604]
  We present a novel framework of continual learning named "Bilevel Continual Learning" (BCL) Our experiments on continual learning benchmarks demonstrate the efficacy of the proposed BCL compared to many state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-30T16:00:23Z)
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey [53.73359052511171]
  Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. We present a framework for curriculum learning (CL) in RL, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals.
  arXiv  Detail & Related papers  (2020-03-10T20:41:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.