A Two-Stage Statistical Framework for Evaluating Associative Interference in Large Language Models

• URL: http://arxiv.org/abs/2606.14117v1
• Date: Fri, 12 Jun 2026 04:56:57 GMT
• Title: A Two-Stage Statistical Framework for Evaluating Associative Interference in Large Language Models
• Abstract summary: We adapt the Implicit Association Test (IAT) to a controlled, forced-choice framework.<n>We evaluate associative interference across three large language models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly evaluated for bias using adaptations of human psychological paradigms, yet methodological limitations-particularly the conflation of refusal behavior with task performance-have hindered clear interpretation. Here, we adapt the Implicit Association Test (IAT) to a controlled, forced-choice framework and introduce a two-stage modeling approach that separates response compliance from task-consistent classification. Across three contemporary LLMs (Claude Sonnet-4, Gemini 2.5 Pro, and GPT-5), we evaluate associative interference, defined as reduced task-consistency in incongruent relative to congruent conditions. While compliance with the structured response format was uniformly high, interference effects varied substantially across models and domains. Claude Sonnet-4 exhibited strong interference in the Gender--Career domain (DeltaP = 0.086, 95% CrI [0.026, 0.173]) and smaller but credible effects in Gender--Science. Gemini 2.5 Pro showed attenuated interference, and GPT-5 exhibited minimal or no detectable interference across domains. These findings demonstrate that IAT-style associative asymmetries are not a universal property of LLMs, but instead depend on model-specific characteristics. By isolating interference from compliance and modeling item-level variability, this study provides a principled framework for evaluating structured response patterns in LLMs. The results highlight the importance of model-specific assessment and suggest that associative interference can be substantially mitigated in modern systems.

Related papers

• Epistemic Stance Flexibility Probing: Measuring Prompt-Conditioned Register Shift in Large Language Models [0.17188280334580197]
  A language model may be asked what experts believe about a contested claim or what it believes about the claim itself.<n>We introduce ESFP, a behavioral benchmark that treats the contrast between externally attributed and self-attributed prompts as the fundamental unit of measurement.
  arXiv  Detail & Related papers  (2026-07-14T13:10:30Z)
• Seirênes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning [56.48520300004217]
  We present Seirnes, a self-play RL framework that transforms contextual interference into an internal training signal.<n>A single model is trained to both construct plausible yet distracting contexts that expose its own reasoning blind spots.<n>By pitting these competing objectives against each other, Seirnes compels the model to move beyond superficial pattern matching.
  arXiv  Detail & Related papers  (2026-05-12T06:58:35Z)
• Characterizing the Consistency of the Emergent Misalignment Persona [20.023580689198248]
  Fine-tuning large language models on narrowly misaligned data generalizes to broadly misaligned behavior, termed emergent misalignment (EM)<n>We characterize the consistency of the EM persona by fine-tuning Qwen 2.5 32B Instruct on six narrowly misaligned domains.<n>Our results reveal two distinct patterns: coherent-persona models, in which harmful behavior and self-reported misalignment are coupled, and inverted-persona models, which produce harmful outputs while identifying as aligned AI systems.
  arXiv  Detail & Related papers  (2026-04-30T16:26:53Z)
• How Independent are Large Language Models? A Statistical Framework for Auditing Behavioral Entanglement and Reweighting Verifier Ensembles [46.63622714488747]
  Shared pretraining data, distillation, and alignment pipelines can induce hidden behavioral dependencies, latent entanglement.<n>In practice, this manifests as correlated reasoning patterns and synchronized failures.<n>We develop a statistical framework for auditing behavioral entanglement among black-box language models.
  arXiv  Detail & Related papers  (2026-04-08T23:32:06Z)
• On the Paradoxical Interference between Instruction-Following and Task Solving [50.75960598434753]
  Instruction following aims to align Large Language Models (LLMs) with human intent by specifying explicit constraints on how tasks should be performed.<n>We reveal a counterintuitive phenomenon: instruction following can paradoxically interfere with LLMs' task-solving capability.<n>We propose a metric, SUSTAINSCORE, to quantify the interference of instruction following with task solving.
  arXiv  Detail & Related papers  (2026-01-29T17:48:56Z)
• Interpretable Few-Shot Image Classification via Prototypical Concept-Guided Mixture of LoRA Experts [79.18608192761512]
  Self-Explainable Models (SEMs) rely on Prototypical Concept Learning (PCL) to enable their visual recognition processes more interpretable.<n>We propose a Few-Shot Prototypical Concept Classification framework that mitigates two key challenges under low-data regimes: parametric imbalance and representation misalignment.<n>Our approach consistently outperforms existing SEMs by a notable margin, with 4.2%-8.7% relative gains in 5-way 5-shot classification.
  arXiv  Detail & Related papers  (2025-06-05T06:39:43Z)
• Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence [46.548276232795466]
  Polysemanticity is pervasive in language models and remains a major challenge for interpretation and model behavioral control.<n>We map the polysemantic topology of two small models to identify feature pairs that are semantically unrelated yet exhibit interference within models.<n>We intervene at four loci (prompt, token, feature, neuron) and measure induced shifts in the next-token prediction distribution, uncovering polysemantic structures that expose a systematic vulnerability in these models.
  arXiv  Detail & Related papers  (2025-05-16T18:20:42Z)
• A NotSo Simple Way to Beat Simple Bench [0.0]
  This paper presents a novel framework for enhancing reasoning capabilities in large language models (LLMs)<n>We propose a multi-step prompting strategy coupled with global consistency checks to improve model accuracy and robustness.<n>Our results reveal model-specific strengths: Claude excels in maintaining logical consistency, while GPT-4o exhibits exploratory creativity but struggles with ambiguous prompts.
  arXiv  Detail & Related papers  (2024-12-12T16:04:31Z)
• Quantifying perturbation impacts for large language models [49.1574468325115]
  We introduce Distribution-Based Perturbation Analysis (DBPA), a framework that reformulates perturbation analysis as a frequentist hypothesis testing problem.<n>We demonstrate the effectiveness of DBPA in evaluating perturbation impacts, showing its versatility for perturbation analysis.
  arXiv  Detail & Related papers  (2024-12-01T16:13:09Z)
• Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models [16.328341121232484]
  We apply causal effect estimation strategies to measure the effect of context interventions. We investigate robustness to irrelevant changes and sensitivity to impactful changes of Transformers.
  arXiv  Detail & Related papers  (2024-04-03T10:22:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.