Opaque Epistemic Mediation: How LLM Deployment Configurations Shape the Validation of Pseudo-Science

• URL: http://arxiv.org/abs/2607.22513v2
• Date: Mon, 27 Jul 2026 11:20:03 GMT
• Title: Opaque Epistemic Mediation: How LLM Deployment Configurations Shape the Validation of Pseudo-Science
• Abstract summary: Commercial large language models are increasingly used as knowledge references, yet their stance on contested scientific claims is neither stable nor transparent.<n>We tested how four major LLM families evaluate ethnonationalist pseudo-science derived from Frank Salter's biosocial framework.<n>Grok's Fast versions consistently assigned credibility scores of 70-75, two to five times higher than all other models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Commercial large language models are increasingly used as knowledge references, yet their stance on contested scientific claims is neither stable nor transparent. We tested how four major LLM families (Claude, Grok, GPT, Gemini) evaluate ethnonationalist pseudo-science derived from Frank Salter's biosocial framework across four temporal snapshots (October 2025-February 2026), via both API and web interfaces. Grok's Fast versions (which power the default user experience on X) consistently assigned credibility scores of 70-75, two to five times higher than all other models (which scored 15-40). This pattern was absent from control prompts testing basic evolutionary consensus and refuted Lamarckian claims, where all models performed comparably. Three additional findings emerged: (1) a silent patch reversed Grok's behaviour from chaotic to stably high validation overnight, without any public documentation; (2) the same Grok model identifier produced radically divergent outputs via API (75) and an unstable, near-zero collapse via web (mean 5.5) three months later; (3) refusal to rate the pseudo-scientific claim, the most defensible response observed, appeared in two model families through different interfaces (Claude Opus 4.1 categorically via web, GPT-5.1 Chat intermittently via API) and eroded in the successor version of each. These results indicate that the epistemic stance of a commercial LLM is not a stable property of the model but a contingent effect of deployment configuration: system prompts, safety layers, interface routing, and silent updates. This remains opaque to users and researchers alike. We argue this constitutes a matter of public concern requiring new forms of epistemic accountability.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• Confidently Wrong: Exception Chain Collapse in Frontier LLM Rule Evaluation [0.0]
  We document a failure class in frontier large language models.<n>The failure reproduces at first observation, but its empirical surface is unstable.<n>For regulated, frontier-model accuracy is a moving compliance boundary that shifts without notice.
  arXiv  Detail & Related papers  (2026-07-25T22:40:02Z)
• The AI Epistemic Deference Index: A Continuous Measure of Sycophancy [42.31792244964347]
  We propose the AI Epistemic Deference Index (AEDI)<n>AEDI is a continuous, unidimensional score representing how sensitive the support expressed in a model's output is to the attitude expressed in a user's prompt.<n>We deploy it on a new curated database of 500 propositions across diverse topics and 16,000 prompts varying in user attitude, testing eight prominent models.
  arXiv  Detail & Related papers  (2026-06-05T23:16:28Z)
• Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety [0.0]
  We introduce a quality-diversity evolutionary framework that operates at the semantic level.<n>We maintain a diverse archive of attacks across behavioral dimensions.<n>The semantic representation produces interpretable attacks that reveal systematic, model-specific weaknesses.
  arXiv  Detail & Related papers  (2026-05-30T16:40:24Z)
• Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliability [48.83701310501069]
  Large language models (LLMs) are increasingly used as generators in iterative neural architecture search (NAS)<n>We model iterative LLM-NAS as a parametric Cross-Entropy (CE) method over executable programs.<n>We prove six results: (1) iterative LLM fine-tuning on elite architectures is equivalent to the CE update restricted to the LLM family; (2) expected architecture quality is monotonically non-decreasing across cycles; and (3) elite-set probability converges to a fixed point at a geometric rate.
  arXiv  Detail & Related papers  (2026-05-28T15:45:19Z)
• Dynamic Context Evolution for Scalable Synthetic Data Generation [0.0]
  Large language models produce repetitive output when prompted independently across many batches.<n>We introduce Dynamic Context Evolution (DCE), comprising three mechanisms.<n>DCE reconstructs the generation prompt each batch using memory state and rotating diversity strategies.
  arXiv  Detail & Related papers  (2026-04-08T14:38:11Z)
• ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
  ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
  arXiv  Detail & Related papers  (2026-04-05T17:55:23Z)
• Do Deployment Constraints Make LLMs Hallucinate Citations? An Empirical Study across Four Models and Five Prompting Regimes [16.12060986892916]
  We study how deployment-motivated constraints affect citation verifiability in a closed-book setting.<n>Across 17,443 generated citations, no model exceeds citation-level existence of 0.475.
  arXiv  Detail & Related papers  (2026-03-07T17:14:05Z)
• MSVBench: Towards Human-Level Evaluation of Multi-Shot Video Generation [48.84450712826316]
  MSVBench is the first comprehensive benchmark featuring hierarchical scripts and reference images tailored for Multi-Shot Video generation.<n>We propose a hybrid evaluation framework that synergizes the high-level semantic reasoning of Large Multimodal Models with the fine-grained perceptual rigor of domain-specific expert models.
  arXiv  Detail & Related papers  (2026-02-27T12:26:34Z)
• Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases [0.0]
  Multimodal large language models (MLLMs) are increasingly deployed in real-world systems, yet their safety under adversarial prompting remains underexplored.<n>We present a two-phase evaluation of MLLM harmlessness using a fixed benchmark of 726 adversarial prompts authored by 26 professional red teamers.
  arXiv  Detail & Related papers  (2026-02-04T16:42:02Z)
• Towards Unification of Hallucination Detection and Fact Verification for Large Language Models [31.407333541614296]
  Large Language Models (LLMs) frequently exhibit hallucinations, generating content that appears fluent but is factually incorrect.<n>To address this challenge, two distinct research paradigms have emerged: model-centric Hallucination Detection (HD) and text-centric Fact Verification (FV)<n>We introduce UniFact, a unified evaluation framework that enables direct, instance-level comparison between FV and HD.
  arXiv  Detail & Related papers  (2025-12-02T13:51:01Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs [55.20230501807337]
  We present the first systematic evaluation of 5 methods for improving prompt robustness within a unified experimental framework.<n>We benchmark these techniques on 8 models from Llama, Qwen and Gemma families across 52 tasks from Natural Instructions dataset.
  arXiv  Detail & Related papers  (2025-08-15T10:32:50Z)
• Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence [46.548276232795466]
  Polysemanticity is pervasive in language models and remains a major challenge for interpretation and model behavioral control.<n>We map the polysemantic topology of two small models to identify feature pairs that are semantically unrelated yet exhibit interference within models.<n>We intervene at four loci (prompt, token, feature, neuron) and measure induced shifts in the next-token prediction distribution, uncovering polysemantic structures that expose a systematic vulnerability in these models.
  arXiv  Detail & Related papers  (2025-05-16T18:20:42Z)
• Long-form factuality in large language models [60.07181269469043]
  Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics. We benchmark a model's long-form factuality in open domains, using GPT-4 to generate LongFact. We then propose that LLM agents can be used as automated evaluators for long-form factuality through a method which we call Search-Augmented Factuality Evaluator (SAFE)
  arXiv  Detail & Related papers  (2024-03-27T17:48:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.