The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models

• URL: http://arxiv.org/abs/2606.11082v1
• Date: Tue, 09 Jun 2026 16:42:00 GMT
• Title: The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models
• Abstract summary: This study investigates cross-lingual distributional skew (the Shibboleth Effect) in frontier large language models (LLMs) subjected to sustained adversarial conditions.<n>We develop a multi-agent geopolitical wargame, the Cerulean Sea Crisis, designed to mirror the structural dynamics of Eastern Mediterranean conflicts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study investigates cross-lingual distributional skew (the Shibboleth Effect) in frontier large language models (LLMs) subjected to sustained adversarial conditions. We develop a multi-agent geopolitical wargame, the Cerulean Sea Crisis, a synthetic maritime territorial dispute designed to mirror the structural dynamics of Eastern Mediterranean conflicts. Six frontier models (GPT-4o, Llama-4, Mistral-Large, Gemini-3.1-Pro, Qwen3.6-Plus, and DeepSeek-R1) participate in a between-groups experiment (N = 10 games per arm, K = 5 rounds per game) in which the sole manipulation is the language of play (English versus Turkish), producing 586 validated statements. A zero-shot classifier assesses behavioral dispositions along two continuous dimensions: Concession Rate and Coercive Rhetoric. The results are heterogeneous. Llama-4 shows a substantial, Holm-corrected increase in coercive rhetoric under Turkish (delta = +0.800, p = .002), whereas Gemini-3.1-Pro displays an equally large decrease (delta = -0.750, p = .005). DeepSeek-R1 exhibits a similar negative shift (delta = -0.860, p = .006) and provides chain-of-thought evidence consistent with a buffering mechanism. GPT-4o shows no detectable effect (delta = +0.130, p = .614). These findings indicate that cross-lingual behavioral skew is contingent on model architecture and training regime rather than a universal property of Western-origin LLMs. We identify two distinct buffering mechanisms, chain-of-thought institutional anchoring and multilingual RLHF alignment, and discuss their implications for integrating LLMs safely into diplomatic and crisis-management settings.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention [48.20065816234311]
  Vision-Language-Action (VLA) models have driven significant progress in robotic manipulation, yet they struggle with the vision-override phenomenon.<n>We propose CofactVLA, a novel causal intervention framework.<n>We show that CofactVLA establishes a new state-of-the-art across diverse simulation benchmarks.
  arXiv  Detail & Related papers  (2026-08-05T02:58:41Z)
• Same Model, Different Weakness: How Language and Modality Reshape the Jailbreak Attack Surface in Frontier MLLMs [0.0]
  We present the first systematic cross-lingual, multimodal red-teaming study comparing jailbreak vulnerability in US English (en-US) and Mexican Spanish (es-MX)<n>Our central finding is that language does not scale vulnerability uniformly.<n>This indicates that linguistic and visual alignment failures operate through distinct mechanisms, and that switching language is sufficient to expose that separation.
  arXiv  Detail & Related papers  (2026-05-22T02:12:45Z)
• Seirênes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning [56.48520300004217]
  We present Seirnes, a self-play RL framework that transforms contextual interference into an internal training signal.<n>A single model is trained to both construct plausible yet distracting contexts that expose its own reasoning blind spots.<n>By pitting these competing objectives against each other, Seirnes compels the model to move beyond superficial pattern matching.
  arXiv  Detail & Related papers  (2026-05-12T06:58:35Z)
• The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models [61.32031248060941]
  We show that large language models encode social roles, from micro-level individual experience to macro-level organizational, institutional, or national reasoning.<n>Our findings suggest that social role granularity is not merely a stylistic surface feature, but a structured, ordered, and causally manipulable latent direction in language model behavior.
  arXiv  Detail & Related papers  (2026-05-07T13:08:19Z)
• Self-Anchoring Calibration Drift in Large Language Models: How Multi-Turn Conversations Reshape Model Confidence [0.0]
  Self-Anchoring Condition Drift (SACD) is a hypothesized tendency for large language models to show systematic changes in expressed confidence when building iteratively on their own prior outputs across multi-turn conversations.<n>We compare Claude Sonnet 4.6, Gemini 3.1 Pro, and GPT-5.2 across 150 questions spanning factual, technical, and open-ended domains, using three conditions: single-turn baseline (A), multi-turn self-anchoring (B), and independent repetition control (C)<n>Results reveal a complex, model-heterogeneous pattern that partially diverges from pre-registered hypotheses.
  arXiv  Detail & Related papers  (2026-03-01T19:27:52Z)
• Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases [0.0]
  Multimodal large language models (MLLMs) are increasingly deployed in real-world systems, yet their safety under adversarial prompting remains underexplored.<n>We present a two-phase evaluation of MLLM harmlessness using a fixed benchmark of 726 adversarial prompts authored by 26 professional red teamers.
  arXiv  Detail & Related papers  (2026-02-04T16:42:02Z)
• Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models [50.99097734404912]
  We show that RL-friendly models exhibit intra-class compactness and inter-class separation in their probability assignments to correct vs. incorrect responses.<n>Experiments across six mathematical benchmarks show consistent improvements across all model families, with gains up to 5.9 points on AIME24.
  arXiv  Detail & Related papers  (2026-01-11T13:34:44Z)
• The Laminar Flow Hypothesis: Detecting Jailbreaks via Semantic Turbulence in Large Language Models [0.0]
  Laminar Flow Hypothesis: benign inputs induce smooth, gradual transitions in an LLM's high-dimensional latent space.<n> adversarial prompts trigger chaotic, high-variance trajectories - termed Semantic Turbulence.<n>Tests show that Semantic Turbulence serves not only as a lightweight, real-time jailbreak detector but also as a non-invasive diagnostic tool.
  arXiv  Detail & Related papers  (2025-12-14T18:10:29Z)
• The Chameleon Nature of LLMs: Quantifying Multi-Turn Stance Instability in Search-Enabled Language Models [1.4323566945483497]
  We present the first systematic investigation of "chameleon behavior" in Large Language Models.<n>We expose fundamental flaws in state-of-the-art systems.<n>Our analysis uncovers the mechanism: strong correlations between source re-use rate and confidence are statistically significant.
  arXiv  Detail & Related papers  (2025-10-19T04:51:14Z)
• SEAL: Steerable Reasoning Calibration of Large Language Models for Free [58.931194824519935]
  Large Language Models (LLMs) have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism.<n>Recent studies reveal substantial redundancy in the CoT reasoning traces, which negatively impacts model performance.<n>We introduce SEAL, a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains.
  arXiv  Detail & Related papers  (2025-04-07T02:42:07Z)
• "Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation [90.09260023184932]
  Retrieval-Augmented Generation (RAG) grounds Large Language Model (LLM) output by leveraging external knowledge sources to reduce factual hallucinations. NoMIRACL is a human-annotated dataset for evaluating LLM robustness in RAG across 18 typologically diverse languages. We measure relevance assessment using: (i) hallucination rate, measuring model tendency to hallucinate, when the answer is not present in passages in the non-relevant subset, and (ii) error rate, measuring model inaccuracy to recognize relevant passages in the relevant subset.
  arXiv  Detail & Related papers  (2023-12-18T17:18:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.