Self-Instruct: Aligning Language Models with Self-Generated Instructions
- URL: http://arxiv.org/abs/2212.10560v2
- Date: Thu, 25 May 2023 23:50:07 GMT
- Title: Self-Instruct: Aligning Language Models with Self-Generated Instructions
- Authors: Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith,
Daniel Khashabi, Hannaneh Hajishirzi
- Abstract summary: Self-Instruct is a framework for improving the instruction-following capabilities of pretrained language models.
Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model.
For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin.
- Score: 76.42871502364697
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large "instruction-tuned" language models (i.e., finetuned to respond to
instructions) have demonstrated a remarkable ability to generalize zero-shot to
new tasks. Nevertheless, they depend heavily on human-written instruction data
that is often limited in quantity, diversity, and creativity, therefore
hindering the generality of the tuned model. We introduce Self-Instruct, a
framework for improving the instruction-following capabilities of pretrained
language models by bootstrapping off their own generations. Our pipeline
generates instructions, input, and output samples from a language model, then
filters invalid or similar ones before using them to finetune the original
model. Applying our method to the vanilla GPT3, we demonstrate a 33% absolute
improvement over the original model on Super-NaturalInstructions, on par with
the performance of InstructGPT-001, which was trained with private user data
and human annotations. For further evaluation, we curate a set of
expert-written instructions for novel tasks, and show through human evaluation
that tuning GPT3 with Self-Instruct outperforms using existing public
instruction datasets by a large margin, leaving only a 5% absolute gap behind
InstructGPT-001. Self-Instruct provides an almost annotation-free method for
aligning pre-trained language models with instructions, and we release our
large synthetic dataset to facilitate future studies on instruction tuning. Our
code and data are available at https://github.com/yizhongw/self-instruct.
Related papers
- REInstruct: Building Instruction Data from Unlabeled Corpus [49.82314244648043]
We propose REInstruct, a method to automatically build instruction data from an unlabeled corpus.
By training Llama-7b on a combination of 3k seed data and 32k synthetic data from REInstruct, fine-tuned model achieves a 65.41% win rate on AlpacaEval leaderboard against text-davinci-003.
arXiv Detail & Related papers (2024-08-20T09:05:03Z) - Self-Alignment with Instruction Backtranslation [162.02529653768096]
We present a method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions.
Our approach, named instruction backtranslation, starts with a language model finetuned on a small amount of seed data, and a given web corpus.
arXiv Detail & Related papers (2023-08-11T17:47:54Z) - Evaluating the Zero-shot Robustness of Instruction-tuned Language Models [23.488398944358643]
We find that using novel (unobserved) but appropriate instruction phrasings consistently degrades model performance.
We propose a simple method to mitigate this issue by introducing soft prompt'' embedding parameters.
We show that this method consistently improves the robustness of instruction-tuned models.
arXiv Detail & Related papers (2023-06-20T03:48:51Z) - LongForm: Effective Instruction Tuning with Reverse Instructions [74.14035528786997]
We introduce the LongForm-C dataset, which is created by reverse instructions.
We generate instructions via LLMs for human-written corpus examples using reverse instructions.
Our models outperform 10x larger language models without instruction tuning on tasks such as story/recipe generation and long-form question answering.
arXiv Detail & Related papers (2023-04-17T17:36:35Z) - Unnatural Instructions: Tuning Language Models with (Almost) No Human
Labor [48.116843121810135]
We introduce Unnatural Instructions: a large dataset of creative and diverse instructions, collected with virtually no human labor.
We collect 64,000 examples by prompting a language model with three seed examples of instructions and eliciting a fourth.
This set is then expanded by prompting the model to rephrase each instruction, creating a total of approximately 240,000 examples of instructions, inputs, and outputs.
arXiv Detail & Related papers (2022-12-19T18:21:00Z) - Training language models to follow instructions with human feedback [29.590666996229206]
We show an avenue for aligning language models with user intent by fine-tuning with human feedback.
InstructGPT models show improvements in truthfulness and reductions in toxic output generation.
arXiv Detail & Related papers (2022-03-04T07:04:42Z) - Finetuned Language Models Are Zero-Shot Learners [67.70352207685558]
We show that instruction tuning boosts zero-shot performance on unseen tasks.
We take a 137B parameter pretrained language model and instruction-tune it on over 60 NLP tasks verbalized via natural language instruction templates.
We evaluate this instruction-tuned model, which we call FLAN, on unseen task types.
arXiv Detail & Related papers (2021-09-03T17:55:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.