If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models

• URL: http://arxiv.org/abs/2510.08388v1
• Date: Thu, 09 Oct 2025 16:12:10 GMT
• Title: If Probable, Then Acceptable? Understanding Conditional Acceptability Judgments in Large Language Models
• Abstract summary: Conditional acceptability refers to how plausible a conditional statement is perceived to be.<n>It influences how individuals interpret implications, assess arguments, and make decisions based on hypothetical scenarios.<n>It remains unclear how large language models judge the $textitacceptability$ of such statements.
• Score: 37.930280449304696
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Conditional acceptability refers to how plausible a conditional statement is perceived to be. It plays an important role in communication and reasoning, as it influences how individuals interpret implications, assess arguments, and make decisions based on hypothetical scenarios. When humans evaluate how acceptable a conditional "If A, then B" is, their judgments are influenced by two main factors: the $\textit{conditional probability}$ of $B$ given $A$, and the $\textit{semantic relevance}$ of the antecedent $A$ given the consequent $B$ (i.e., whether $A$ meaningfully supports $B$). While prior work has examined how large language models (LLMs) draw inferences about conditional statements, it remains unclear how these models judge the $\textit{acceptability}$ of such statements. To address this gap, we present a comprehensive study of LLMs' conditional acceptability judgments across different model families, sizes, and prompting strategies. Using linear mixed-effects models and ANOVA tests, we find that models are sensitive to both conditional probability and semantic relevance-though to varying degrees depending on architecture and prompting style. A comparison with human data reveals that while LLMs incorporate probabilistic and semantic cues, they do so less consistently than humans. Notably, larger models do not necessarily align more closely with human judgments.

Related papers

• Sensitivity to Subjective Expected Utility Maximization: A Methodological Study, with an Illustrative Application to LLM Decision-Making [0.0]
  Vehicle is a softmax choice model with a sensitivity parameter $$ on SEU-valued alternatives.<n>The contribution is a sequence of identifiability results for $$ and for belief and utility parameters $(, )$, validated in Stansberg.<n>A two-by-two application runs end-to-end on real choice data, detecting a structured comparative $$ effect sampling in two of four cells.
  arXiv  Detail & Related papers  (2026-07-08T00:29:18Z)
• Incoherent Values? Probing LLM Preferences Through Parametric Variation [0.1753733541634709]
  It is hard to trust someone with incoherent values.<n>Some think of Large Language Models as merely text generators with no evaluative core.<n>We show that models given time to reason are less incoherent than those with thinking disabled.
  arXiv  Detail & Related papers  (2026-06-19T05:07:34Z)
• Evaluating Bivariate Causal Statements Based on Mutual Compatibility [7.594559816880284]
  We develop methods for evaluating causal statements over a set of $n$ variables.<n>We introduce a compatibility score that quantifies this notion of plausibility without relying on the faithfulness assumption.<n>We give theoretical and empirical evidence that both scores can successfully distinguish correct from incorrect causal statements in generic settings.
  arXiv  Detail & Related papers  (2026-05-29T19:15:09Z)
• Regression-Based Estimation of Causal Effects in the Presence of Selection Bias and Confounding [52.1068936424622]
  We consider the problem of estimating the expected causal effect $E[Y|do(X)]$ for a target variable $Y$ when treatment $X$ is set by intervention.<n>In settings without selection bias or confounding, $E[Y|do(X)] = E[Y|X]$, which can be estimated using standard regression methods.<n>We propose a framework that incorporates both selection bias and confounding.
  arXiv  Detail & Related papers  (2025-03-26T13:43:37Z)
• Let's Think Var-by-Var: Large Language Models Enable Ad Hoc Probabilistic Reasoning [15.568698101627088]
  We propose to extract common sense from large language models (LLMs)<n>We focus our investigation on $textitguesstimation$ questions such as "How much are Airbnb listings in Newark, NJ?"<n>Our framework answers such a question by an $textitad hoc$ probabilistic model.
  arXiv  Detail & Related papers  (2024-12-03T01:53:06Z)
• Partial Identifiability and Misspecification in Inverse Reinforcement Learning [64.13583792391783]
  The aim of Inverse Reinforcement Learning is to infer a reward function $R$ from a policy $pi$. This paper provides a comprehensive analysis of partial identifiability and misspecification in IRL.
  arXiv  Detail & Related papers  (2024-11-24T18:35:46Z)
• QUITE: Quantifying Uncertainty in Natural Language Text in Bayesian Reasoning Scenarios [15.193544498311603]
  We present QUITE, a dataset of real-world Bayesian reasoning scenarios with categorical random variables and complex relationships. We conduct an extensive set of experiments, finding that logic-based models outperform out-of-the-box large language models on all reasoning types. Our results provide evidence that neuro-symbolic models are a promising direction for improving complex reasoning.
  arXiv  Detail & Related papers  (2024-10-14T12:44:59Z)
• Mind the Gap: A Causal Perspective on Bias Amplification in Prediction & Decision-Making [58.06306331390586]
  We introduce the notion of a margin complement, which measures how much a prediction score $S$ changes due to a thresholding operation. We show that under suitable causal assumptions, the influences of $X$ on the prediction score $S$ are equal to the influences of $X$ on the true outcome $Y$.
  arXiv  Detail & Related papers  (2024-05-24T11:22:19Z)
• Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models [50.15455336684986]
  We evaluate the effectiveness of LogProbs and basic prompting to measure semantic plausibility. We find that LogProbs offers a more reliable measure of semantic plausibility than direct zero-shot prompting. We conclude that, even in the era of prompt-based evaluations, LogProbs constitute a useful metric of semantic plausibility.
  arXiv  Detail & Related papers  (2024-03-21T22:08:44Z)
• Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification [72.08225446179783]
  Inverse reinforcement learning aims to infer an agent's preferences from their behaviour. To do this, we need a behavioural model of how $pi$ relates to $R$. We analyse how sensitive the IRL problem is to misspecification of the behavioural model.
  arXiv  Detail & Related papers  (2024-03-11T16:09:39Z)
• Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Models [44.56781176879151]
  State-of-the-art Natural Language Inference (NLI) models are sensitive towards minor semantics preserving surface-form variations. We show that semantic sensitivity causes performance degradations of $12.92%$ and $23.71%$ average over $textbfin-$ and $textbfout-of-$ domain settings.
  arXiv  Detail & Related papers  (2024-01-25T14:47:05Z)
• Misspecification in Inverse Reinforcement Learning [80.91536434292328]
  The aim of Inverse Reinforcement Learning (IRL) is to infer a reward function $R$ from a policy $pi$. One of the primary motivations behind IRL is to infer human preferences from human behaviour. This means that they are misspecified, which raises the worry that they might lead to unsound inferences if applied to real-world data.
  arXiv  Detail & Related papers  (2022-12-06T18:21:47Z)
• The Projected Covariance Measure for assumption-lean variable significance testing [3.8936058127056357]
  A simple but common approach is to specify a linear model, and then test whether the regression coefficient for $X$ is non-zero. We study the problem of testing the model-free null of conditional mean independence, i.e. that the conditional mean of $Y$ given $X$ and $Z$ does not depend on $X$. We propose a simple and general framework that can leverage flexible nonparametric or machine learning methods, such as additive models or random forests.
  arXiv  Detail & Related papers  (2022-11-03T17:55:50Z)
• Probabilistic Variational Causal Approach in Observational Studies [0.0]
  We introduce a new causal methodology that accounts for the rarity and frequency of events in observational studies based on their relevance to the underlying problem.<n>We propose a direct causal effect metric called the Probabilistic vAriational Causal Effect (PACE) and its variations adhering to certain postulates applicable to both non-binary and binary treatments.
  arXiv  Detail & Related papers  (2022-08-12T13:34:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.