A Mechanistic Understanding of Pronoun Fidelity in LLMs

• URL: http://arxiv.org/abs/2606.16407v1
• Date: Mon, 15 Jun 2026 08:45:53 GMT
• Title: A Mechanistic Understanding of Pronoun Fidelity in LLMs
• Abstract summary: We study whether three mechanisms -- group entity binding (G), recency bias (R), and stereotypical bias (S) -- are causally implemented across several SOTA language models.<n>No single mechanism fully explains model behaviour, but a combination of the three consistently accounts for 99.5%1-9.<n>In sum, pronoun fidelity arises from competition between simultaneously active causal subspaces.
• Score: 21.554337243825884
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Faithful and robust pronoun use is important for fair and coherent generations, yet large language models largely fail when multiple referents use different pronouns. To study the interplay of reasoning, repetition, and bias in this task, prior work relies exclusively on behavioural approaches, which may not reflect a model's internal workings. Therefore, we provide a mechanistic, model-internal perspective on pronoun fidelity, testing whether three mechanisms -- group entity binding (G), recency bias (R), and stereotypical bias (S) -- are causally implemented across several SOTA language models. Using Boundless Distributed Alignment Search, we find all three coexist as causal subspaces distributed across network depth. No single mechanism fully explains model behaviour, but a combination of the three consistently accounts for 91-99.5%. An attention head analysis further reveals two competing copying routes; group binding and stereotype share a localized concept-level route that retrieves a bound occupation-pronoun unit, while recency uses a distributed token-level route that repeats surface forms. In sum, pronoun fidelity arises from competition between simultaneously active causal subspaces.

Related papers

• Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement [66.67463557274358]
  Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes.<n>We introduce Confidence-guided Concept Mining (CBCM) to identify reliable spurious attributes without attribute annotations.<n>We propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches.
  arXiv  Detail & Related papers  (2026-06-23T05:28:47Z)
• DisPOSE: Projected Polystochastic Diffusion for Self-Supervised Multi-View 3D Human Pose Estimation [58.47973015036709]
  DisPOSE is a self-supervised framework that approximates the inherently discrete multi-view person-assignment problem.<n>By employing differentiable Sinkhorn projections during denoising, our model learns to guide solutions toward valid and feasible assignments.<n>The proposed approach outperforms current state-of-the-art self-supervised methods on standard datasets.
  arXiv  Detail & Related papers  (2026-06-05T16:14:39Z)
• Peer Identity Bias in Multi-Agent LLM Evaluation: An Empirical Study Using the TRUST Democratic Discourse Analysis Pipeline [0.0]
  We provide the first systematic measurement of identity-dependent scoring bias across all active identity exposure channels in TRUST.<n>We find that single-channel anonymization produces near-zero bias effects, because individual channels act in opposite directions and cancel each other out.<n>One tested model exhibits baseline sycophancy two to three times higher than the others and near-zero deliberative conflict on ideological topics.
  arXiv  Detail & Related papers  (2026-04-24T19:32:01Z)
• The Illusion of Superposition? A Principled Analysis of Latent Thinking in Language Models [15.12701445445687]
  We investigate whether language models actually leverage superposition when reasoning using latent CoTs.<n>We find that only models trained from scratch exhibit signs of using superposition.<n>Our results offer a unified explanation for when and why superposition arises in continuous chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2026-04-07T18:59:32Z)
• UniG2U-Bench: Do Unified Models Advance Multimodal Understanding? [50.92401586025528]
  Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear.<n>We introduce UniG2U-Bench, a comprehensive benchmark categorizing generation-to-understanding (G2U) evaluation into 7 regimes and 30 subtasks.
  arXiv  Detail & Related papers  (2026-03-03T18:36:16Z)
• Generation Space Size: Understanding and Calibrating Open-Endedness of LLM Generations [30.476953783731307]
  We argue that effective generation space size (GSS) is the set of semantically distinct outputs a model considers for a prompt.<n>We present GSSBench, a task suite of prompt pairs with ground-truth GSS relationships to assess different metrics.<n>We find that hallucination detection metrics, particularly EigenScore, consistently outperform standard diversity and uncertainty quantification metrics.
  arXiv  Detail & Related papers  (2025-10-14T16:31:34Z)
• Escaping Plato's Cave: JAM for Aligning Independently Trained Vision and Language Models [30.07172193932125]
  We show that the Joint Autoencoder Modulator (JAM) induces alignment even across independently trained representations.<n>Our findings offer theoretical insight into the structure of shared semantics and practical guidance for transforming generalist unimodal foundations into specialist multimodal models.
  arXiv  Detail & Related papers  (2025-07-01T21:43:50Z)
• A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models [58.32070787537946]
  Chain-of-thought (CoT) reasoning enhances performance of large language models.<n>We present the first comprehensive study of CoT faithfulness in large vision-language models.
  arXiv  Detail & Related papers  (2025-05-29T18:55:05Z)
• Rethinking the Two-Stage Framework for Grounded Situation Recognition [61.93345308377144]
  Grounded Situation Recognition is an essential step towards "human-like" event understanding. Existing GSR methods resort to a two-stage framework: predicting the verb in the first stage and detecting the semantic roles in the second stage. We propose a novel SituFormer for GSR which consists of a Coarse-to-Fine Verb Model (CFVM) and a Transformer-based Noun Model (TNM)
  arXiv  Detail & Related papers  (2021-12-10T08:10:56Z)
• Multi-sense embeddings through a word sense disambiguation process [2.2344764434954256]
  Most Suitable Sense. (MSSA) disambiguates and annotates each word by its specific sense, considering the semantic effects of its context. We test our approach on six different benchmarks for the word similarity task, showing that our approach can produce state-of-the-art results.
  arXiv  Detail & Related papers  (2021-01-21T16:22:34Z)
• Transformer-GCRF: Recovering Chinese Dropped Pronouns with General Conditional Random Fields [54.03719496661691]
  We present a novel framework that combines the strength of Transformer network with General Conditional Random Fields (GCRF) to model the dependencies between pronouns in neighboring utterances. Results on three Chinese conversation datasets show that the Transformer-GCRF model outperforms the state-of-the-art dropped pronoun recovery models.
  arXiv  Detail & Related papers  (2020-10-07T07:06:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.