Enriching a Model's Notion of Belief using a Persistent Memory

• URL: http://arxiv.org/abs/2104.08401v1
• Date: Fri, 16 Apr 2021 23:09:11 GMT
• Title: Enriching a Model's Notion of Belief using a Persistent Memory
• Authors: Nora Kassner, Oyvind Tafjord, Hinrich Schutze, Peter Clark
• Abstract summary: Pretrained language models (PTLMs) can produce inconsistent answers to questions when probed. It can be hard to identify what the model actually "believes" about the world. Our goal is to reduce this problem, so systems are more globally consistent and accurate in their answers.
• Score: 20.60798513220516
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although pretrained language models (PTLMs) have been shown to contain significant amounts of world knowledge, they can still produce inconsistent answers to questions when probed, even after using specialized training techniques to reduce inconsistency. As a result, it can be hard to identify what the model actually "believes" about the world. Our goal is to reduce this problem, so systems are more globally consistent and accurate in their answers. Our approach is to add a memory component - a BeliefBank - that records a model's answers, and two mechanisms that use it to improve consistency among beliefs. First, a reasoning component - a weighted SAT solver - improves consistency by flipping answers that significantly clash with others. Second, a feedback component re-queries the model but using known beliefs as context. We show that, in a controlled experimental setting, these two mechanisms improve both accuracy and consistency. This is significant as it is a first step towards endowing models with an evolving memory, allowing them to construct a more coherent picture of the world.

Related papers

• Disentangling Memory and Reasoning Ability in Large Language Models [97.26827060106581]
  We propose a new inference paradigm that decomposes the complex inference process into two distinct and clear actions. Our experiment results show that this decomposition improves model performance and enhances the interpretability of the inference process.
  arXiv  Detail & Related papers  (2024-11-20T17:55:38Z)
• Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? [65.43882564649721]
  Large language models (LLMs) have demonstrated impressive capabilities, but still suffer from inconsistency issues. We develop the ConsisEval benchmark, where each entry comprises a pair of questions with a strict order of difficulty. We analyze the potential for improvement in consistency by relative consistency score.
  arXiv  Detail & Related papers  (2024-06-18T17:25:47Z)
• Language Models with Rationality [57.37201135072838]
  Large language models (LLMs) are proficient at question-answering (QA) It is not always clear how (or even if) an answer follows from their latent "beliefs"
  arXiv  Detail & Related papers  (2023-05-23T17:04:25Z)
• Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning [26.715242799194908]
  We show how a question-answering system can show how its answers are implied by its own internal beliefs via a systematic chain of reasoning. Our approach is to combine a trained backward-chaining model, capable of generating a set of premises entailing an answer hypothesis, with a verifier that checks that the model itself believes those premises. To our knowledge, this is the first system to generate multistep chains that are both faithful (the answer follows from the reasoning) and truthful (the chain reflects the system's own internal beliefs)
  arXiv  Detail & Related papers  (2022-10-21T19:51:56Z)
• Measuring and Narrowing the Compositionality Gap in Language Models [116.5228850227024]
  We measure how often models can correctly answer all sub-problems but not generate the overall solution. We present a new method, self-ask, that further improves on chain of thought.
  arXiv  Detail & Related papers  (2022-10-07T06:50:23Z)
• Towards Teachable Reasoning Systems [29.59387051046722]
  We develop a teachable reasoning system for question-answering (QA) Our approach is three-fold: First, generated chains of reasoning show how answers are implied by the system's own internal beliefs. Second, users can interact with the explanations to identify erroneous model beliefs and provide corrections. Third, we augment the model with a dynamic memory of such corrections.
  arXiv  Detail & Related papers  (2022-04-27T17:15:07Z)
• Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs [76.6325846350907]
  Dennett (1995) famously argues that even thermostats have beliefs, on the view that a belief is simply an informational state decoupled from any motivational state. In this paper, we discuss approaches to detecting when models have beliefs about the world, and we improve on methods for updating model beliefs to be more truthful.
  arXiv  Detail & Related papers  (2021-11-26T18:33:59Z)
• BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief [20.60798513220516]
  It can be hard to identify what the model actually "believes" about the world, making it susceptible to inconsistent behavior and simple errors. Our approach is to embed a PTLM in a broader system that includes an evolving, symbolic memory of beliefs. We show that, in a controlled experimental setting, these two mechanisms result in more consistent beliefs in the overall system.
  arXiv  Detail & Related papers  (2021-09-29T21:04:27Z)
• Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models [62.28551903638434]
  We measure the impact of three different adaptation methods on the generalization and accuracy of models. Experiments with two models show that fine-tuning performs best, by learning both the content and the structure of the task, but suffers from overfitting and limited generalization to novel answers. We observe that alternative adaptation methods like prefix-tuning have comparable accuracy, but generalize better to unseen answers and are more robust to adversarial splits.
  arXiv  Detail & Related papers  (2021-09-07T03:13:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.