Implicit Chain of Thought Reasoning via Knowledge Distillation

• URL: http://arxiv.org/abs/2311.01460v1
• Date: Thu, 2 Nov 2023 17:59:49 GMT
• Title: Implicit Chain of Thought Reasoning via Knowledge Distillation
• Authors: Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, Stuart Shieber
• Abstract summary: Instead of explicitly producing the chain of thought reasoning steps, we use the language model's internal hidden states to perform implicit reasoning. We find that this approach enables solving tasks previously not solvable without explicit chain-of-thought, at a speed comparable to no chain-of-thought.
• Score: 58.80851216530288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To augment language models with the ability to reason, researchers usually prompt or finetune them to produce chain of thought reasoning steps before producing the final answer. However, although people use natural language to reason effectively, it may be that LMs could reason more effectively with some intermediate computation that is not in natural language. In this work, we explore an alternative reasoning approach: instead of explicitly producing the chain of thought reasoning steps, we use the language model's internal hidden states to perform implicit reasoning. The implicit reasoning steps are distilled from a teacher model trained on explicit chain-of-thought reasoning, and instead of doing reasoning "horizontally" by producing intermediate words one-by-one, we distill it such that the reasoning happens "vertically" among the hidden states in different layers. We conduct experiments on a multi-digit multiplication task and a grade school math problem dataset and find that this approach enables solving tasks previously not solvable without explicit chain-of-thought, at a speed comparable to no chain-of-thought.

Related papers

• Contrastive Chain-of-Thought Prompting [74.10511560147293]
  We propose contrastive chain of thought to enhance language model reasoning. Compared to the conventional chain of thought, our approach provides both valid and invalid reasoning demonstrations. Our experiments on reasoning benchmarks demonstrate that contrastive chain of thought can serve as a general enhancement of chain-of-thought prompting.
  arXiv  Detail & Related papers  (2023-11-15T18:54:01Z)
• Preventing Language Models From Hiding Their Reasoning [0.0]
  Large language models (LLMs) often benefit from intermediate steps of reasoning to generate answers to complex problems. In this work, we focus on one potential way intermediate steps of reasoning could be unfaithful: encoded reasoning. We show that language models can be trained to make use of encoded reasoning to get higher performance without the user understanding the intermediate steps of reasoning.
  arXiv  Detail & Related papers  (2023-10-27T22:02:29Z)
• Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic [19.476840373850653]
  Large language models show hallucinations as their reasoning procedures are unconstrained by logical principles. We propose LoT (Logical Thoughts), a self-improvement prompting framework that leverages principles rooted in symbolic logic. Experimental evaluations conducted on language tasks in diverse domains, including arithmetic, commonsense, symbolic, causal inference, and social problems, demonstrate the efficacy of enhanced reasoning by logic.
  arXiv  Detail & Related papers  (2023-09-23T11:21:12Z)
• Boosting Language Models Reasoning with Chain-of-Knowledge Prompting [18.326858925174605]
  Chain-of-Knowledge (CoK) prompting aims at eliciting explicit pieces of knowledge evidence in the form of structure triple. Benefiting from CoK, we additionally introduce a F2-Verification method to estimate the reliability of the reasoning chains. Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.
  arXiv  Detail & Related papers  (2023-06-10T12:42:36Z)
• Deductive Verification of Chain-of-Thought Reasoning [22.79166959432764]
  Large Language Models (LLMs) benefit from Chain-of-Thought prompting in performing various reasoning tasks. While CoT allows models to produce more comprehensive reasoning processes, its emphasis on intermediate reasoning steps can inadvertently introduce hallucinations and accumulated errors. We propose Natural Program, a natural language-based deductive reasoning format.
  arXiv  Detail & Related papers  (2023-06-06T17:18:56Z)
• Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners [75.85554779782048]
  Large Language Models (LLMs) have excited the natural language and machine learning community over recent years. Despite of numerous successful applications, the underlying mechanism of such in-context capabilities still remains unclear. In this work, we hypothesize that the learned textitsemantics of language tokens do the most heavy lifting during the reasoning process.
  arXiv  Detail & Related papers  (2023-05-24T07:33:34Z)
• Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations [118.0818807474809]
  Abductive reasoning aims to find plausible explanations for an event. Existing approaches for abductive reasoning in natural language processing often rely on manually generated annotations for supervision. This work proposes an approach for abductive commonsense reasoning that exploits the fact that only a subset of explanations is correct for a given context.
  arXiv  Detail & Related papers  (2023-05-24T01:35:10Z)
• Learning to Reason and Memorize with Self-Notes [51.17609489687686]
  Large language models have been shown to struggle with multi-step reasoning. We propose a simple method for solving both of these problems by allowing the model to take Self-Notes.
  arXiv  Detail & Related papers  (2023-05-01T14:02:48Z)
• Chain of Thought Prompting Elicits Reasoning in Large Language Models [56.811278668446825]
  This paper explores the ability of language models to generate a coherent chain of thought. Experiments show that inducing a chain of thought via prompting can enable sufficiently large language models to better perform reasoning tasks.
  arXiv  Detail & Related papers  (2022-01-28T02:33:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.