Self-Alignment with Instruction Backtranslation
- URL: http://arxiv.org/abs/2308.06259v3
- Date: Tue, 12 Mar 2024 05:22:46 GMT
- Title: Self-Alignment with Instruction Backtranslation
- Authors: Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke
Zettlemoyer, Jason Weston, Mike Lewis
- Abstract summary: We present a method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions.
Our approach, named instruction backtranslation, starts with a language model finetuned on a small amount of seed data, and a given web corpus.
- Score: 162.02529653768096
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a scalable method to build a high quality instruction following
language model by automatically labelling human-written text with corresponding
instructions. Our approach, named instruction backtranslation, starts with a
language model finetuned on a small amount of seed data, and a given web
corpus. The seed model is used to construct training examples by generating
instruction prompts for web documents (self-augmentation), and then selecting
high quality examples from among these candidates (self-curation). This data is
then used to finetune a stronger model. Finetuning LLaMa on two iterations of
our approach yields a model that outperforms all other LLaMa-based models on
the Alpaca leaderboard not relying on distillation data, demonstrating highly
effective self-alignment.
Related papers
- Kun: Answer Polishment for Chinese Self-Alignment with Instruction
Back-Translation [51.43576926422795]
Kun is a novel approach for creating high-quality instruction-tuning datasets for large language models (LLMs) without relying on manual annotations.
We leverage unlabelled data from diverse sources such as Wudao, Wanjuan, and SkyPile to generate a substantial dataset of over a million Chinese instructional data points.
arXiv Detail & Related papers (2024-01-12T09:56:57Z) - Self-Supervised Representation Learning for Online Handwriting Text
Classification [0.8594140167290099]
We propose the novel Part of Stroke Masking (POSM) as a pretext task for pretraining models to extract informative representations from the online handwriting of individuals in English and Chinese languages.
To evaluate the quality of the extracted representations, we use both intrinsic and extrinsic evaluation methods.
The pretrained models are fine-tuned to achieve state-of-the-art results in tasks such as writer identification, gender classification, and handedness classification.
arXiv Detail & Related papers (2023-10-10T14:07:49Z) - LongForm: Effective Instruction Tuning with Reverse Instructions [43.7029933201002]
We introduce the LongForm-C dataset, which is created by reverse instructions.
First we select a diverse set of human-written documents from corpora such as C4 and Wikipedia.
We generate instructions via LLMs for human-written corpus examples using reverse instructions.
arXiv Detail & Related papers (2023-04-17T17:36:35Z) - Self-Instruct: Aligning Language Models with Self-Generated Instructions [76.42871502364697]
Self-Instruct is a framework for improving the instruction-following capabilities of pretrained language models.
Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model.
For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin.
arXiv Detail & Related papers (2022-12-20T18:59:19Z) - Revisiting Self-Training for Few-Shot Learning of Language Model [61.173976954360334]
Unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model.
In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM.
arXiv Detail & Related papers (2021-10-04T08:51:36Z) - Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting.
Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking.
We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
arXiv Detail & Related papers (2020-10-24T11:55:28Z) - Exemplar-Controllable Paraphrasing and Translation using Bitext [57.92051459102902]
We adapt models from prior work to be able to learn solely from bilingual text (bitext)
Our single proposed model can perform four tasks: controlled paraphrase generation in both languages and controlled machine translation in both language directions.
arXiv Detail & Related papers (2020-10-12T17:02:50Z) - Abstractive Text Summarization based on Language Model Conditioning and
Locality Modeling [4.525267347429154]
We train a Transformer-based neural model on the BERT language model.
In addition, we propose a new method of BERT-windowing, which allows chunk-wise processing of texts longer than the BERT window size.
The results of our models are compared to a baseline and the state-of-the-art models on the CNN/Daily Mail dataset.
arXiv Detail & Related papers (2020-03-29T14:00:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.