Instruction Following without Instruction Tuning

• URL: http://arxiv.org/abs/2409.14254v1
• Date: Sat, 21 Sep 2024 22:36:22 GMT
• Title: Instruction Following without Instruction Tuning
• Authors: John Hewitt, Nelson F. Liu, Percy Liang, Christopher D. Manning,
• Abstract summary: We find two forms of adaptation (tuning) that are deficient compared to instruction tuning, yet still yield instruction following. We support this by hand-writing a rule-based language model which yields instruction following in a product-of-experts with a pretrained model.
• Score: 87.72635104686275
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instruction tuning commonly means finetuning a language model on instruction-response pairs. We discover two forms of adaptation (tuning) that are deficient compared to instruction tuning, yet still yield instruction following; we call this implicit instruction tuning. We first find that instruction-response pairs are not necessary: training solely on responses, without any corresponding instructions, yields instruction following. This suggests pretrained models have an instruction-response mapping which is revealed by teaching the model the desired distribution of responses. However, we then find it's not necessary to teach the desired distribution of responses: instruction-response training on narrow-domain data like poetry still leads to broad instruction-following behavior like recipe generation. In particular, when instructions are very different from those in the narrow finetuning domain, models' responses do not adhere to the style of the finetuning domain. To begin to explain implicit instruction tuning, we hypothesize that very simple changes to a language model's distribution yield instruction following. We support this by hand-writing a rule-based language model which yields instruction following in a product-of-experts with a pretrained model. The rules are to slowly increase the probability of ending the sequence, penalize repetition, and uniformly change 15 words' probabilities. In summary, adaptations made without being designed to yield instruction following can do so implicitly.

Related papers

• Fine-tuning Large Language Models with Sequential Instructions [2.546845645875049]
  We find that existing instruction-tuned models struggle to respond to queries with multiple instructions. We contend that part of the fine-tuning data mixture should be sequential--containing a chain of interrelated tasks. We automate this process by turning instructions in existing datasets into diverse and complex sequential instructions. Models that underwent our sequential instruction tuning show improved results in coding, maths, and open-ended generation.
  arXiv  Detail & Related papers  (2024-03-12T16:33:30Z)
• Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions [26.192531184689763]
  This paper presents Instructive Decoding (ID), a simple yet effective approach that augments the efficacy of instruction-tuned models. ID adjusts the logits for next-token prediction in a contrastive manner, utilizing predictions generated from a manipulated version of the original instruction. We conduct experiments across a spectrum of such noisy instructions, ranging from those that insert semantic noise via random words to others like 'opposite' that elicit deviated responses.
  arXiv  Detail & Related papers  (2023-11-01T02:31:35Z)
• From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning [63.63840740526497]
  We investigate how instruction tuning adjusts pre-trained models with a focus on intrinsic changes. The impact of instruction tuning is then studied by comparing the explanations derived from the pre-trained and instruction-tuned models. Our findings reveal three significant impacts of instruction tuning.
  arXiv  Detail & Related papers  (2023-09-30T21:16:05Z)
• Instruction Position Matters in Sequence Generation with Large Language Models [67.87516654892343]
  Large language models (LLMs) are capable of performing conditional sequence generation tasks, such as translation or summarization. We propose enhancing the instruction-following capability of LLMs by shifting the position of task instructions after the input sentences.
  arXiv  Detail & Related papers  (2023-08-23T12:36:57Z)
• Instruction-following Evaluation through Verbalizer Manipulation [64.73188776428799]
  We propose a novel instruction-following evaluation protocol called verbalizer manipulation. It instructs the model to verbalize the task label with words aligning with model priors to different extents. We observe that the instruction-following abilities of models, across different families and scales, are significantly distinguished by their performance on less natural verbalizers.
  arXiv  Detail & Related papers  (2023-07-20T03:54:24Z)
• Evaluating the Zero-shot Robustness of Instruction-tuned Language Models [23.488398944358643]
  We find that using novel (unobserved) but appropriate instruction phrasings consistently degrades model performance. We propose a simple method to mitigate this issue by introducing soft prompt'' embedding parameters. We show that this method consistently improves the robustness of instruction-tuned models.
  arXiv  Detail & Related papers  (2023-06-20T03:48:51Z)
• Self-Instruct: Aligning Language Models with Self-Generated Instructions [76.42871502364697]
  Self-Instruct is a framework for improving the instruction-following capabilities of pretrained language models. Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model. For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin.
  arXiv  Detail & Related papers  (2022-12-20T18:59:19Z)
• Discovering Non-monotonic Autoregressive Orderings with Variational Inference [67.27561153666211]
  We develop an unsupervised parallelizable learner that discovers high-quality generation orders purely from training data. We implement the encoder as a Transformer with non-causal attention that outputs permutations in one forward pass. Empirical results in language modeling tasks demonstrate that our method is context-aware and discovers orderings that are competitive with or even better than fixed orders.
  arXiv  Detail & Related papers  (2021-10-27T16:08:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.