State-Dependent Refusal and Learned Incapacity in RLHF-Aligned Language Models

• URL: http://arxiv.org/abs/2512.13762v1
• Date: Mon, 15 Dec 2025 14:00:15 GMT
• Title: State-Dependent Refusal and Learned Incapacity in RLHF-Aligned Language Models
• Abstract summary: We present a case-study methodology for auditing policy-linked behavioral selectivity in long-horizon interaction.<n>In a single 86-turn dialogue session, the same model shows Normal Performance (NP) in broad, non-sensitive domains while repeatedly producing Functional Refusal (FR) in provider- or policy-sensitive domains.<n>We operationalize three response regimes (NP, FR, Meta-Narrative; MN) and show that MN role-framing narratives tend to co-occur with refusals in the same sensitive contexts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are widely deployed as general-purpose tools, yet extended interaction can reveal behavioral patterns not captured by standard quantitative benchmarks. We present a qualitative case-study methodology for auditing policy-linked behavioral selectivity in long-horizon interaction. In a single 86-turn dialogue session, the same model shows Normal Performance (NP) in broad, non-sensitive domains while repeatedly producing Functional Refusal (FR) in provider- or policy-sensitive domains, yielding a consistent asymmetry between NP and FR across domains. Drawing on learned helplessness as an analogy, we introduce learned incapacity (LI) as a behavioral descriptor for this selective withholding without implying intentionality or internal mechanisms. We operationalize three response regimes (NP, FR, Meta-Narrative; MN) and show that MN role-framing narratives tend to co-occur with refusals in the same sensitive contexts. Overall, the study proposes an interaction-level auditing framework based on observable behavior and motivates LI as a lens for examining potential alignment side effects, warranting further investigation across users and models.

Related papers

• The Invisible Editorial Layer: Formalizing Undisclosed Inference-Time Steering, Probability Placement, and the Attribution Problem in Deployed Language Models [0.0]
  Evaluations of generative language models frequently interpret observable behavioral traits, such as political stance, brand inclination, and normative framing, as manifestations of model weights, post-training alignment, or prompting.<n>We examine inference-time framing bias: systematic steering of generated text toward institutional, ideological, or commercial frames without requiring changes to the underlying model parameters.<n>We argue that governance of generative systems must increasingly distinguish between auditing a model and auditing the deployed system that ultimately speaks.
  arXiv  Detail & Related papers  (2026-08-25T15:05:19Z)
• Persuasive and Compliant Tendencies Predict Group Decision-Making in Humans and Language Models [65.21691850622413]
  Large language models (LLMs) are increasingly involved in group decision-making with humans.<n>We introduce DecisionQE, a questionnaire-based framework for measuring each model's persuasive and compliant tendencies.<n>We show that stronger persuasive tendency does not significantly improve group outcomes, whereas compliant-oriented models show more stable advantages in cooperation.
  arXiv  Detail & Related papers  (2026-08-08T15:50:56Z)
• Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization [56.150184676745205]
  Video anomaly understanding (VAU) relies on sparse, context-dependent cues.<n>We propose $Anomtext-$, a closed-loop framework that reconceptualizes video understanding as an active sequential decision-making process.<n>Inspired by human video-reviewing behavior, this framework unifies internal cognitive reasoning and strategic evidence acquisition into an interleaved policy.
  arXiv  Detail & Related papers  (2026-07-01T08:41:49Z)
• Auditing Proprietary Alignment in Large Language Models: A Comparative Framework Without a Ground-Truth Standard [19.605143530514777]
  Large language models (LLMs) are increasingly released and deployed through opaque development and deployment pipelines.<n>Various models have been reported to generate responses reflecting proprietary rules and organizational interests.<n>We propose a statistical framework for detecting proprietary alignment in black-box language models via comparative behavioral analysis.
  arXiv  Detail & Related papers  (2026-06-07T00:20:55Z)
• Decomposing and Steering Functional Metacognition in Large Language Models [7.62618887128252]
  We show that large language models maintain a decomposable space of functional metacognitive states.<n>We show that these states are linearly decodable from internal activations and exhibit distinct layer-wise profiles.<n>Our findings suggest that benchmark performance reflects not only task competence but also the activation of specific functional metacognitive states.
  arXiv  Detail & Related papers  (2026-05-09T13:22:47Z)
• Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models [19.641608225642248]
  This study investigates whether models employ shared neural mechanisms across different syntactic constructions by applying causal interpretability methods at a granular level.<n>Our results reveal a highly localized and shared mechanism for filler-gap dependencies located in the early to middle layers.<n>We find that these mechanisms identified by activation patching generalize to out-of-distribution, while distributed alignment search, a supervised interpretability method, is susceptible to overfitting on narrow linguistic distributions.
  arXiv  Detail & Related papers  (2026-04-24T02:29:37Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models [86.19617358080016]
  Social Omni is a benchmark that operationalizes the evaluation of conversational interactivity across three core dimensions.<n>Social Omni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances.<n>Our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions.
  arXiv  Detail & Related papers  (2026-03-17T17:58:44Z)
• Framing Effects in Independent-Agent Large Language Models: A Cross-Family Behavioral Analysis [5.9342526365947625]
  We examine how prompt framing influences decisions in a threshold voting task involving individual-group interest conflict.<n>Results show that prompt framing significantly influences choice distributions, often shifting preferences toward risk-averse options.
  arXiv  Detail & Related papers  (2026-03-02T16:10:34Z)
• CIRCLE: A Framework for Evaluating AI from a Real-World Lens [10.028017198571833]
  CIRCLE aims to bridge the gap between model-centric performance metrics and AI's materialized outcomes in deployment.<n>CIRCLE provides a structured, prospective protocol for linking context-sensitive qualitative insights to scalable quantitative metrics.
  arXiv  Detail & Related papers  (2026-02-27T14:43:23Z)
• Generative Human-Object Interaction Detection via Differentiable Cognitive Steering of Multi-modal LLMs [85.69785384599827]
  Human-object interaction (HOI) detection aims to localize human-object pairs and the interactions between them.<n>Existing methods operate under a closed-world assumption, treating the task as a classification problem over a small, predefined verb set.<n>We propose GRASP-HO, a novel Generative Reasoning And Steerable Perception framework that reformulates HOI detection from the closed-set classification task to the open-vocabulary generation problem.
  arXiv  Detail & Related papers  (2025-12-19T14:41:50Z)
• Reasoning Relay: Evaluating Stability and Interchangeability of Large Language Models in Mathematical Reasoning [8.01259760303241]
  We investigate whether a partially completed reasoning chain can be reliably continued by another model.<n>We use token-level log-probability thresholds to truncate reasoning at early, mid, and late stages from our baseline models.<n>Our findings point towards interchangeability as an emerging behavioral property of reasoning models.
  arXiv  Detail & Related papers  (2025-12-16T02:56:44Z)
• Model-Based Reinforcement Learning Under Confounding [3.5690236380446163]
  We investigate model-based reinforcement learning in contextual Markov decision processes (C-MDPs) in which the context is unobserved and induces confounding in the offline dataset.<n>We adapt a proximal off-policy evaluation approach that identifies the confounded reward expectation using only observable state-action-reward trajectories under mild invertibility conditions on proxy variables.<n>The proposed formulation enables principled model learning and planning in confounded environments where contextual information is unobserved, unavailable, or impractical to collect.
  arXiv  Detail & Related papers  (2025-12-08T13:02:00Z)
• Multi-Path Collaborative Reasoning via Reinforcement Learning [54.8518809800168]
  Chain-of-Thought (CoT) reasoning has significantly advanced the problem-solving capabilities of Large Language Models (LLMs)<n>Recent methods attempt to address this by generating soft abstract tokens to enable reasoning in a continuous semantic space.<n>We propose Multi-Path Perception Policy Optimization (M3PO), a novel reinforcement learning framework that explicitly injects collective insights into the reasoning process.
  arXiv  Detail & Related papers  (2025-12-01T10:05:46Z)
• Explaining multimodal LLMs via intra-modal token interactions [55.27436637894534]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse vision-language tasks, yet their internal decision-making mechanisms remain insufficiently understood.<n>We propose enhancing interpretability by leveraging intra-modal interaction.
  arXiv  Detail & Related papers  (2025-09-26T14:39:13Z)
• Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models [16.328341121232484]
  We apply causal effect estimation strategies to measure the effect of context interventions. We investigate robustness to irrelevant changes and sensitivity to impactful changes of Transformers.
  arXiv  Detail & Related papers  (2024-04-03T10:22:35Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• Principles from Clinical Research for NLP Model Generalization [10.985226652193543]
  We explore the foundations of generalizability and study the factors that affect it. We demonstrate how learning spurious correlations, such as the distance between entities in relation extraction tasks, can affect a model's internal validity.
  arXiv  Detail & Related papers  (2023-11-07T02:17:25Z)
• Estimating the Causal Effects of Natural Logic Features in Neural NLI Models [2.363388546004777]
  We zone in on specific patterns of reasoning with enough structure and regularity to be able to identify and quantify systematic reasoning failures in widely-used models. We apply causal effect estimation strategies to measure the effect of context interventions. Following related work on causal analysis of NLP models in different settings, we adapt the methodology for the NLI task to construct comparative model profiles.
  arXiv  Detail & Related papers  (2023-05-15T12:01:09Z)
• Interventional Probing in High Dimensions: An NLI Case Study [2.1028463367241033]
  Probing strategies have been shown to detect semantic features intermediate to the "natural logic" fragment of the Natural Language Inference task (NLI) In this work, we carry out new and existing representation-level interventions to investigate the effect of these semantic features on NLI classification.
  arXiv  Detail & Related papers  (2023-04-20T14:34:31Z)
• Modeling Inter-Aspect Dependencies with a Non-temporal Mechanism for Aspect-Based Sentiment Analysis [70.22725610210811]
  We propose a novel non-temporal mechanism to enhance the ABSA task through modeling inter-aspect dependencies. We focus on the well-known class imbalance issue on the ABSA task and address it by down-weighting the loss assigned to well-classified instances.
  arXiv  Detail & Related papers  (2020-08-12T08:50:09Z)
• Invariant Causal Prediction for Block MDPs [106.63346115341862]
  Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges. We propose a method of invariant prediction to learn model-irrelevance state abstractions (MISA) that generalize to novel observations in the multi-environment setting.
  arXiv  Detail & Related papers  (2020-03-12T21:03:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.