Related papers: Are language models rational? The case of coherence norms and belief revision

Are language models rational? The case of coherence norms and belief revision

URL: http://arxiv.org/abs/2406.03442v2
Date: Sat, 10 Aug 2024 21:55:08 GMT
Title: Are language models rational? The case of coherence norms and belief revision
Authors: Thomas Hofweber, Peter Hase, Elias Stengel-Eskin, Mohit Bansal,
Abstract summary: We consider logical coherence norms as well as coherence norms tied to the strength of belief in language models. We argue that rational norms tied to coherence do apply to some language models, but not to others.
Score: 63.78798769882708
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Do norms of rationality apply to machine learning models, in particular language models? In this paper we investigate this question by focusing on a special subset of rational norms: coherence norms. We consider both logical coherence norms as well as coherence norms tied to the strength of belief. To make sense of the latter, we introduce the Minimal Assent Connection (MAC) and propose a new account of credence, which captures the strength of belief in language models. This proposal uniformly assigns strength of belief simply on the basis of model internal next token probabilities. We argue that rational norms tied to coherence do apply to some language models, but not to others. This issue is significant since rationality is closely tied to predicting and explaining behavior, and thus it is connected to considerations about AI safety and alignment, as well as understanding model behavior more generally.

Related papers

Rulebreakers Challenge: Revealing a Blind Spot in Large Language Models' Reasoning with Formal Logic [3.0648414540406703]
This study introduces the concept of "rulebreakers", which refers to instances where logical entailment diverges from factually acceptable inference. We present RULEBREAKERS, a novel dataset for evaluating Large Language Models' ability to distinguish between rulebreakers and non-rulebreakers.
arXiv Detail & Related papers (2024-10-21T20:48:16Z)
AURA: Natural Language Reasoning for Aleatoric Uncertainty in Rationales [0.0]
Rationales behind answers not only explain model decisions but boost language models to reason well on complex reasoning tasks. It is non-trivial to estimate the degree to which the rationales are faithful enough to encourage model performance. We propose how to deal with imperfect rationales causing aleatoric uncertainty.
arXiv Detail & Related papers (2024-02-22T07:12:34Z)
NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly [61.77957329364812]
We introduce a framework for addressing the novel task of conversation-grounded multi-lingual, multi-cultural norm discovery. NormSAGE elicits knowledge about norms through directed questions representing the norm discovery task and conversation context. It further addresses the risk of language model hallucination with a self-verification mechanism ensuring that the norms discovered are correct.
arXiv Detail & Related papers (2022-10-16T18:30:05Z)
Causal Inference Principles for Reasoning about Commonsense Causality [93.19149325083968]
Commonsense causality reasoning aims at identifying plausible causes and effects in natural language descriptions that are deemed reasonable by an average person. Existing work usually relies on deep language models wholeheartedly, and is potentially susceptible to confounding co-occurrences. Motivated by classical causal principles, we articulate the central question of CCR and draw parallels between human subjects in observational studies and natural languages. We propose a novel framework, ROCK, to Reason O(A)bout Commonsense K(C)ausality, which utilizes temporal signals as incidental supervision.
arXiv Detail & Related papers (2022-01-31T06:12:39Z)
Learning Symbolic Rules for Reasoning in Quasi-Natural Language [74.96601852906328]
We build a rule-based system that can reason with natural language input but without the manual construction of rules. We propose MetaQNL, a "Quasi-Natural" language that can express both formal logic and natural language sentences. Our approach achieves state-of-the-art accuracy on multiple reasoning benchmarks.
arXiv Detail & Related papers (2021-11-23T17:49:00Z)
Rationales for Sequential Predictions [117.93025782838123]
Sequence models are a critical component of modern NLP systems, but their predictions are difficult to explain. We consider model explanations though rationales, subsets of context that can explain individual model predictions. We propose an efficient greedy algorithm to approximate this objective.
arXiv Detail & Related papers (2021-09-14T01:25:15Z)
Probabilistic modelling of rational communication with conditionals [0.0]
We take a probabilistic approach to pragmatic reasoning about conditionals. We show that our model uniformly explains a number of inferences attested in the literature.
arXiv Detail & Related papers (2021-05-12T08:21:25Z)
Measuring Association Between Labels and Free-Text Rationales [60.58672852655487]
In interpretable NLP, we require faithful rationales that reflect the model's decision-making process for an explained instance. We demonstrate that pipelines, existing models for faithful extractive rationalization on information-extraction style tasks, do not extend as reliably to "reasoning" tasks requiring free-text rationales. We turn to models that jointly predict and rationalize, a class of widely used high-performance models for free-text rationalization whose faithfulness is not yet established.
arXiv Detail & Related papers (2020-10-24T03:40:56Z)
A Defeasible Calculus for Zetetic Agents [0.0]
We show that zetetic norms can be modeled via defeasible inferences to and from questions. We offer a sequent calculus that accommodates unique features of "erotetic defeat"
arXiv Detail & Related papers (2020-10-11T17:39:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.