Certified Deductive Reasoning with Language Models
- URL: http://arxiv.org/abs/2306.04031v2
- Date: Wed, 8 Nov 2023 01:53:31 GMT
- Title: Certified Deductive Reasoning with Language Models
- Authors: Gabriel Poesia, Kanishk Gandhi, Eric Zelikman, Noah D. Goodman
- Abstract summary: We introduce a class of tools for language models called emphguides, that use state and incremental constraints to guide generation.
A guide can be invoked by the model to constrain its own generation to a set of valid statements.
We show how a general system for logical reasoning can be used as a guide, which we call textscLogicGuide.
- Score: 37.51289654360009
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Language models often achieve higher accuracy when reasoning step-by-step in
complex tasks. However, even when arriving at a correct final answer, their
rationales are often logically unsound or inconsistent. This is a major issue
when reliable reasoning traces are needed, such when fine-tuning on
model-generated reasoning for self-improvement. To tackle these issues, we
introduce a class of tools for language models called \emph{guides}, that use
state and incremental constraints to guide generation. A guide can be invoked
by the model to constrain its own generation to a set of valid statements given
by the tool. In turn, the model's choices can change the guide's state. We show
how a general system for logical reasoning can be used as a guide, which we
call \textsc{LogicGuide}. Given a reasoning problem in natural language, a
model can formalize its assumptions for \textsc{LogicGuide} and guarantee that
its step-by-step reasoning is sound. In experiments on PrOntoQA, ProofWriter
and Syllogism Validity datasets, \textsc{LogicGuide} significantly improves the
performance of GPT-3, GPT-3.5 Turbo and LLaMA (accuracy gains up to 35\%),
while drastically reducing \emph{content effects} -- the interference between
unwanted prior assumptions and reasoning, which humans and language models
suffer from. We then explore bootstrapping GPT-3.5 Turbo and LLaMA using their
own reasoning traces. We find that LogicGuide is critical: by training only on
certified self-generated reasoning, models can self-improve, avoiding learning
from their own hallucinations. Moreover, bootstrapped models enjoy significant
boosts on ReClor, a challenging real-world reasoning dataset, even when not
relying on formalization at inference time.
Related papers
- Preventing Language Models From Hiding Their Reasoning [0.0]
Large language models (LLMs) often benefit from intermediate steps of reasoning to generate answers to complex problems.
In this work, we focus on one potential way intermediate steps of reasoning could be unfaithful: encoded reasoning.
We show that language models can be trained to make use of encoded reasoning to get higher performance without the user understanding the intermediate steps of reasoning.
arXiv Detail & Related papers (2023-10-27T22:02:29Z) - Deductive Verification of Chain-of-Thought Reasoning [22.79166959432764]
Large Language Models (LLMs) benefit from Chain-of-Thought prompting in performing various reasoning tasks.
While CoT allows models to produce more comprehensive reasoning processes, its emphasis on intermediate reasoning steps can inadvertently introduce hallucinations and accumulated errors.
We propose Natural Program, a natural language-based deductive reasoning format.
arXiv Detail & Related papers (2023-06-06T17:18:56Z) - Exposing Attention Glitches with Flip-Flop Language Modeling [55.0688535574859]
This work identifies and analyzes the phenomenon of attention glitches in large language models.
We introduce flip-flop language modeling (FFLM), a family of synthetic benchmarks designed to probe the extrapolative behavior of neural language models.
We find that Transformer FFLMs suffer from a long tail of sporadic reasoning errors, some of which we can eliminate using various regularization techniques.
arXiv Detail & Related papers (2023-06-01T17:44:35Z) - REFINER: Reasoning Feedback on Intermediate Representations [47.36251998678097]
We introduce REFINER, a framework for finetuning language models to generate intermediate inferences.
REFINER works by interacting with a critic model that provides automated feedback on the reasoning.
Empirical evaluations show significant improvements over baseline LMs of comparable scale.
arXiv Detail & Related papers (2023-04-04T15:57:28Z) - ALERT: Adapting Language Models to Reasoning Tasks [43.8679673685468]
ALERT is a benchmark and suite of analyses for assessing language models' reasoning ability.
ALERT provides a test bed to asses any language model on fine-grained reasoning skills.
We find that language models learn more reasoning skills during finetuning stage compared to pretraining state.
arXiv Detail & Related papers (2022-12-16T05:15:41Z) - Discovering Latent Knowledge in Language Models Without Supervision [72.95136739040676]
Existing techniques for training language models can be misaligned with the truth.
We propose directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way.
We show that despite using no supervision and no model outputs, our method can recover diverse knowledge represented in large language models.
arXiv Detail & Related papers (2022-12-07T18:17:56Z) - MetaLogic: Logical Reasoning Explanations with Fine-Grained Structure [129.8481568648651]
We propose a benchmark to investigate models' logical reasoning capabilities in complex real-life scenarios.
Based on the multi-hop chain of reasoning, the explanation form includes three main components.
We evaluate the current best models' performance on this new explanation form.
arXiv Detail & Related papers (2022-10-22T16:01:13Z) - Language Models Are Greedy Reasoners: A Systematic Formal Analysis of
Chain-of-Thought [10.524051272257614]
Large language models (LLMs) have shown remarkable reasoning capabilities given chain-of-thought prompts.
We present a new synthetic question-answering dataset called PrOntoQA, where each example is generated as a synthetic world model.
This allows us to parse the generated chain-of-thought into symbolic proofs for formal analysis.
arXiv Detail & Related papers (2022-10-03T21:34:32Z) - The Unreliability of Explanations in Few-Shot In-Context Learning [50.77996380021221]
We focus on two NLP tasks that involve reasoning over text, namely question answering and natural language inference.
We show that explanations judged as good by humans--those that are logically consistent with the input--usually indicate more accurate predictions.
We present a framework for calibrating model predictions based on the reliability of the explanations.
arXiv Detail & Related papers (2022-05-06T17:57:58Z) - Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason
Over Implicit Knowledge [96.92252296244233]
Large pre-trained language models (LMs) acquire some reasoning capacity, but this ability is difficult to control.
We show that LMs can be trained to reliably perform systematic reasoning combining both implicit, pre-trained knowledge and explicit natural language statements.
Our work paves a path towards open-domain systems that constantly improve by interacting with users who can instantly correct a model by adding simple natural language statements.
arXiv Detail & Related papers (2020-06-11T17:02:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.