Teaching Small Language Models to Reason

• URL: http://arxiv.org/abs/2212.08410v3
• Date: Thu, 1 Jun 2023 12:17:01 GMT
• Title: Teaching Small Language Models to Reason
• Authors: Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, Aliaksei Severyn
• Abstract summary: Chain of thought prompting successfully improves the reasoning capabilities of large language models. We explore the transfer of such reasoning capabilities to models with less than 100 billion parameters via knowledge distillation. Our experiments show that the proposed method improves task performance across arithmetic, commonsense and symbolic reasoning datasets.
• Score: 19.625523231233128
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chain of thought prompting successfully improves the reasoning capabilities of large language models, achieving state of the art results on a range of datasets. However, these reasoning capabilities only appear to emerge in models with a size of over 100 billion parameters. In this paper, we explore the transfer of such reasoning capabilities to models with less than 100 billion parameters via knowledge distillation. Specifically, we finetune a student model on the chain of thought outputs generated by a larger teacher model. Our experiments show that the proposed method improves task performance across arithmetic, commonsense and symbolic reasoning datasets. For example, the accuracy of T5 XXL on GSM8K improves from 8.11% to 21.99% when finetuned on PaLM-540B generated chains of thought.

Related papers

• Building Math Agents with Multi-Turn Iterative Preference Learning [56.71330214021884]
  This paper studies the complementary direct preference learning approach to further improve model performance. Existing direct preference learning algorithms are originally designed for the single-turn chat task. We introduce a multi-turn direct preference learning framework, tailored for this context.
  arXiv  Detail & Related papers  (2024-09-04T02:41:04Z)
• Emergent Abilities in Reduced-Scale Generative Language Models [10.51168925267033]
  Large language models can solve new tasks without task-specific fine-tuning. This ability is considered an emergent ability and is primarily seen in large language models with billions of parameters. This study investigates if such emergent properties are strictly tied to model size or can be demonstrated by smaller models trained on reduced-scale data.
  arXiv  Detail & Related papers  (2024-04-02T18:00:28Z)
• TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise [27.90035459143466]
  We propose TeacherLM-7.1B, capable of annotating relevant fundamentals, chain of thought, and common mistakes for most NLP samples. The model achieved a zero-shot score of 52.3 on MMLU, surpassing most models with over 100B parameters. We will release the TeacherLM series of models and augmented datasets as open-source.
  arXiv  Detail & Related papers  (2023-10-29T14:16:54Z)
• Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA [5.117094291273979]
  Large Language Models (LLMs) have shown outstanding performance across wide range of downstream tasks. We propose Sci-CoT, a two-stage framework that separates the processes of generating rationales and inferring answers. Our 80-million parameter model is able to exceed the performance of BLOOM-176B in the ARC-Easy dataset under the few shot setting.
  arXiv  Detail & Related papers  (2023-08-09T03:18:07Z)
• Text Classification via Large Language Models [63.1874290788797]
  We introduce Clue And Reasoning Prompting (CARP) to address complex linguistic phenomena involved in text classification. Remarkably, CARP yields new SOTA performances on 4 out of 5 widely-used text-classification benchmarks. More importantly, we find that CARP delivers impressive abilities on low-resource and domain-adaptation setups.
  arXiv  Detail & Related papers  (2023-05-15T06:24:45Z)
• Specializing Smaller Language Models towards Multi-Step Reasoning [56.78474185485288]
  We show that abilities can be distilled down from GPT-3.5 ($ge$ 175B) to T5 variants ($le$ 11B) We propose model specialization, to specialize the model's ability towards a target task.
  arXiv  Detail & Related papers  (2023-01-30T08:51:19Z)
• Go-tuning: Improving Zero-shot Learning Abilities of Smaller Language Models [23.818751895205132]
  Go-tuning is a geometry-guided self-supervised learning method. Go-tuning can enable T5-small (80M) competitive zero-shot results compared with large language models, such as T5-XL (3B)
  arXiv  Detail & Related papers  (2022-12-20T17:36:49Z)
• Large Language Models Are Reasoning Teachers [9.290757451344673]
  Fine-tune-CoT is a method that generates reasoning samples from very large teacher models to fine-tune smaller models. We find that Fine-tune-CoT enables substantial reasoning capability in small models, far outperforming prompt-based baselines and even the teacher model in many tasks.
  arXiv  Detail & Related papers  (2022-12-20T08:24:45Z)
• Scaling Instruction-Finetuned Language Models [126.4789306516927]
  Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance. We find that instruction finetuning dramatically improves performance on a variety of model classes.
  arXiv  Detail & Related papers  (2022-10-20T16:58:32Z)
• PaLM: Scaling Language Modeling with Pathways [180.69584031908113]
  We trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM. We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-of-the-art few-shot learning results on hundreds of language understanding and generation benchmarks.
  arXiv  Detail & Related papers  (2022-04-05T16:11:45Z)
• Scaling Language Models: Methods, Analysis & Insights from Training Gopher [83.98181046650664]
  We present an analysis of Transformer-based language model performance across a wide range of model scales. Gains from scale are largest in areas such as reading comprehension, fact-checking, and the identification of toxic language. We discuss the application of language models to AI safety and the mitigation of downstream harms.
  arXiv  Detail & Related papers  (2021-12-08T19:41:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.