STaR: Bootstrapping Reasoning With Reasoning

• URL: http://arxiv.org/abs/2203.14465v1
• Date: Mon, 28 Mar 2022 03:12:15 GMT
• Title: STaR: Bootstrapping Reasoning With Reasoning
• Authors: Eric Zelikman, Yuhuai Wu, Noah D. Goodman
• Abstract summary: "Self-Taught Reason" (STaR) relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples. We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers.
• Score: 39.45372621632046
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating step-by-step "chain-of-thought" rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering. However, inducing language model rationale generation currently requires either constructing massive rationale datasets or sacrificing accuracy by using only few-shot inference. We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning. This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale examples; if the generated answers are wrong, try again to generate a rationale given the correct answer; fine-tune on all the rationales that ultimately yielded correct answers; repeat. We show that STaR significantly improves performance on multiple datasets compared to a model fine-tuned to directly predict final answers, and performs comparably to fine-tuning a 30$\times$ larger state-of-the-art language model on CommensenseQA. Thus, STaR lets a model improve itself by learning from its own generated reasoning.

Related papers

• Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning [89.89857766491475]
  We propose a complex reasoning schema over KG upon large language models (LLMs) We augment the arbitrary first-order logical queries via binary tree decomposition to stimulate the reasoning capability of LLMs. Experiments across widely used datasets demonstrate that LACT has substantial improvements(brings an average +5.5% MRR score) over advanced methods.
  arXiv  Detail & Related papers  (2024-05-02T18:12:08Z)
• Improving Language Model Reasoning with Self-motivated Learning [60.779625789039486]
  textitSelf-motivated Learning framework motivates the model itself to automatically generate rationales on existing datasets. We train a reward model with the rank to evaluate the quality of rationales, and improve the performance of reasoning through reinforcement learning.
  arXiv  Detail & Related papers  (2024-04-10T14:05:44Z)
• AURA: Natural Language Reasoning for Aleatoric Uncertainty in Rationales [0.0]
  Rationales behind answers not only explain model decisions but boost language models to reason well on complex reasoning tasks. It is non-trivial to estimate the degree to which the rationales are faithful enough to encourage model performance. We propose how to deal with imperfect rationales causing aleatoric uncertainty.
  arXiv  Detail & Related papers  (2024-02-22T07:12:34Z)
• Boosting the Power of Small Multimodal Reasoning Models to Match Larger Models with Self-Consistency Training [49.3242278912771]
  Multimodal reasoning is a challenging task that requires models to reason across multiple modalities to answer questions. Existing approaches have made progress by incorporating language and visual modalities into a two-stage reasoning framework. We propose MC-CoT, a self-consistency training strategy that generates multiple rationales and answers, subsequently selecting the most accurate through a voting process.
  arXiv  Detail & Related papers  (2023-11-23T17:09:48Z)
• ZARA: Improving Few-Shot Self-Rationalization for Small Language Models [29.755148112827502]
  We present a novel approach, Zero-shot Augmentation of Rationale-Answer pairs (ZARA), to automatically construct pseudo-parallel data for self-training. ZARA achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric.
  arXiv  Detail & Related papers  (2023-05-12T10:07:12Z)
• Reasoning Circuits: Few-shot Multihop Question Generation with Structured Rationales [11.068901022944015]
  Chain-of-thought rationale generation has been shown to improve performance on multi-step reasoning tasks. We introduce a new framework for applying chain-of-thought inspired structured rationale generation to multi-hop question generation under a very low supervision regime.
  arXiv  Detail & Related papers  (2022-11-15T19:36:06Z)
• Complexity-Based Prompting for Multi-Step Reasoning [72.0057198610614]
  We study the task of prompting large-scale language models to perform multi-step reasoning. A central question is which reasoning examples make the most effective prompts. We propose complexity-based prompting, a simple and effective example selection scheme for multi-step reasoning.
  arXiv  Detail & Related papers  (2022-10-03T05:33:27Z)
• Rationales for Sequential Predictions [117.93025782838123]
  Sequence models are a critical component of modern NLP systems, but their predictions are difficult to explain. We consider model explanations though rationales, subsets of context that can explain individual model predictions. We propose an efficient greedy algorithm to approximate this objective.
  arXiv  Detail & Related papers  (2021-09-14T01:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.