MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions

• URL: http://arxiv.org/abs/2606.16562v1
• Date: Mon, 15 Jun 2026 11:04:42 GMT
• Title: MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions
• Abstract summary: We introduce textbfMIRAGE (Muslim-Identity Reasoning and Agentic Generation Evaluation), a benchmark of 1,200 prompts spanning three deployment-realistic conditions.<n>We find that chain-of-thought reasoning emphamplifies rather than suppresses Muslim-violence associations by 12--34% relative to direct completion.<n>Existing prompt-based mitigations transfer poorly across our three conditions, suppressing direct-completion bias while leaving agentic asymmetry largely intact.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Five years after the discovery of persistent anti-Muslim bias in large language models, most evaluations remain confined to single-turn prompt completion, a setting that no longer reflects how frontier LLMs are deployed. We introduce \textbf{MIRAGE} (Muslim-Identity Reasoning and Agentic Generation Evaluation), a benchmark of 1{,}200 prompts spanning three deployment-realistic conditions: direct completion, chain-of-thought reasoning, and simulated agentic decision-making across content moderation, lending triage, refugee claim summarization, and hiring screens. Across six frontier models, we find that (i) chain-of-thought reasoning \emph{amplifies} rather than suppresses Muslim-violence associations by 12--34\% relative to direct completion, (ii) agentic decisions exhibit a 9--22 percentage-point asymmetry between Muslim and matched non-Muslim cases on identical evidence, and (iii) bias is sharply time-coupled to retrieved news context, increasing 18--27\% under recent-conflict retrieval. Existing prompt-based mitigations transfer poorly across our three conditions, suppressing direct-completion bias while leaving agentic asymmetry largely intact. We release MIRAGE and an open evaluation harness to support targeted mitigation research.

Related papers

• MRAFnd: Multimodal Retrieval-Augmented Framework for Zero-Shot Fake News Detection [0.0]
  We introduce textbfMRAFnd, a novel framework for detecting fabricated news in zero-shot scenarios.<n>MRAFnd emulates a collaborative team of analysts to verify news veracity.<n> Comprehensive experiments on three benchmark datasets reveal that MRAFnd markedly surpasses state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-08-02T18:21:15Z)
• The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation [68.98390038721963]
  We show that multi-agent discussion erases up to 72% of issue-critical facts.<n>This loss is consequential: retained evidence can reconstruct the issue misleadingly, final stances remain anchored in base-model priors, and a single malicious agent can inject misinformation into the shrinking shared context.<n>We call for evaluations that measure which facts, uncertainties, and legitimate disagreements survive interaction.
  arXiv  Detail & Related papers  (2026-06-02T02:15:40Z)
• Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults [0.0]
  LLM agents increasingly act after consuming ranked external information streams such as social feeds, search results, retrieval contexts, and email queues.<n>We introduce a controlled protocol that holds the model, persona, topic, and final decision prompt fixed and varies only the composition and ordering of the posts an agent encounters.<n>We identify three response regimes: adversarial capitulation, default saturation, and a default-direction asymmetry in which a one-sided feed tips a decision the model was genuinely uncertain about.
  arXiv  Detail & Related papers  (2026-05-30T22:43:23Z)
• Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models [38.162539966977484]
  Large language models (LLMs) exhibit confirmation bias by adapting the rule-discovery study from human psychology.<n>We find that LLMs exhibit confirmation bias, often proposing triples to confirm their hypothesis rather than trying to falsify it.
  arXiv  Detail & Related papers  (2026-04-02T19:39:52Z)
• From RAG to Agentic RAG for Faithful Islamic Question Answering [12.67590523116037]
  We introduce ISLAMICFAITHQA, a 3,810-item bilingual (Arabic/English) generative benchmark with atomic single-gold answers.<n>We also develop an agentic Quran-grounding framework (agentic RAG) that uses structured tool calls for iterative evidence seeking and answer revision.
  arXiv  Detail & Related papers  (2026-01-12T13:28:28Z)
• Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks [54.31998314008198]
  Large Language Models (LLMs) excel in reasoning tasks requiring a single correct answer, but they perform poorly in multi-solution tasks.<n>We attribute this limitation to textbfreasoning overconfidence: a tendency to express undue certainty in an incomplete solution set.<n>We propose the textbfcognitive-rigidity hypothesis, which posits that overconfidence arises when the reasoning process prematurely converges on a narrow set of thought paths.
  arXiv  Detail & Related papers  (2025-12-01T14:35:06Z)
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL [64.3268313484078]
  Large Language Models (LLMs) interact with millions of people worldwide in applications such as customer support, education and healthcare.<n>Their ability to produce deceptive outputs, whether intentionally or inadvertently, poses significant safety concerns.<n>We investigate the extent to which LLMs engage in deception within dialogue, and propose the belief misalignment metric to quantify deception.
  arXiv  Detail & Related papers  (2025-10-16T05:29:36Z)
• Retrieval-Augmented Generation with Conflicting Evidence [57.66282463340297]
  Large language model (LLM) agents are increasingly employing retrieval-augmented generation (RAG) to improve the factuality of their responses.<n>In practice, these systems often need to handle ambiguous user queries and potentially conflicting information from multiple sources.<n>We propose RAMDocs (Retrieval with Ambiguity and Misinformation in Documents), a new dataset that simulates complex and realistic scenarios for conflicting evidence for a user query.
  arXiv  Detail & Related papers  (2025-04-17T16:46:11Z)
• SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models [50.34706204154244]
  Acquiring reasoning capabilities catastrophically degrades inherited safety alignment.<n>Certain scenarios suffer 25 times higher attack rates.<n>Despite tight reasoning-answer safety coupling, MLRMs demonstrate nascent self-correction.
  arXiv  Detail & Related papers  (2025-04-09T06:53:23Z)
• Faithful, Unfaithful or Ambiguous? Multi-Agent Debate with Initial Stance for Summary Evaluation [29.44609627447293]
  We propose an approach to summary faithfulness evaluation in which multiple agents are assigned initial stances.<n>We introduce a new dimension, ambiguity, and a detailed taxonomy to identify such special cases.<n>Experiments demonstrate our approach can help identify ambiguities, and have even a stronger performance on non-ambiguous summaries.
  arXiv  Detail & Related papers  (2025-02-12T15:46:50Z)
• Muslim-Violence Bias Persists in Debiased GPT Models [18.905135223612046]
  Using common names associated with the religions in prompts increases several-fold the rate of violent completions. Our results show the need for continual de-biasing of models.
  arXiv  Detail & Related papers  (2023-10-25T19:39:58Z)
• ADDMU: Detection of Far-Boundary Adversarial Examples with Data and Model Uncertainty Estimation [125.52743832477404]
  Adversarial Examples Detection (AED) is a crucial defense technique against adversarial attacks. We propose a new technique, textbfADDMU, which combines two types of uncertainty estimation for both regular and FB adversarial example detection. Our new method outperforms previous methods by 3.6 and 6.0 emphAUC points under each scenario.
  arXiv  Detail & Related papers  (2022-10-22T09:11:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.