The Oracle's Fingerprint: Correlated AI Forecasting Errors and the Limits of Bias Transmission

• URL: http://arxiv.org/abs/2605.00844v1
• Date: Tue, 07 Apr 2026 19:45:40 GMT
• Title: The Oracle's Fingerprint: Correlated AI Forecasting Errors and the Limits of Bias Transmission
• Abstract summary: We show that GPT-4o, Claude, and Gemini exhibit highly correlated forecasting errors on binary prediction questions.<n>In Study 2, we test whether this correlated bias has propagated into human crowd forecasts.<n>In Study 3, we examine whether the category-level pattern of human forecasting errors increasingly resembles the LLM bias fingerprint.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When large language models (LLMs) are consulted as forecasting tools, the independence of individual errors -- the foundation of collective intelligence -- may collapse. We test three conditions necessary for this "epistemic monoculture" to emerge. In Study 1, we show that GPT-4o, Claude, and Gemini exhibit highly correlated forecasting errors on 568 resolved binary prediction questions (mean pairwise error correlation r = 0.77, p < 0.001; r = 0.78 excluding likely-leaked questions), despite being developed independently by different organizations. In Study 2, we test whether this correlated bias has propagated into human crowd forecasts, using a within-question design that tracks community prediction shifts across the ChatGPT launch boundary (November 2022). We find that community forecasts move in the direction predicted by LLMs (r = 0.20, p = 0.007), but this shift is fully explained by rational updating toward ground truth. In Study 3, we examine whether the category-level pattern of human forecasting errors increasingly resembles the LLM bias fingerprint. We find the opposite: pre-ChatGPT human biases already strongly resembled the LLM pattern (r = 0.87), while post-ChatGPT the resemblance weakened (r = -0.28). Together, these findings reveal an epistemic monoculture that is built but not yet activated: three nominally independent AI systems share the same failure modes, amplifying precisely the biases humans already hold.

Related papers

• Not All Relations Are Equal: Relation-Balanced and Calibrated Graph Learning for Provenance-Based Intrusion Detection [66.44527094471619]
  We present RECAL, an unsupervised framework using relation-balanced masked graph learning to better capture rare interaction patterns.<n>On three DARPA E3 datasets, RECAL achieves F1 scores of 99.99%, 99.93%, and 99.99%, outperforming the best baseline on each dataset by 0.88, 0.82, and 0.42 percentage points, respectively.
  arXiv  Detail & Related papers  (2026-09-15T00:31:01Z)
• Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs [0.764671395172401]
  We show that whether mode-collapse, or its opposite, occurs depends on the specific model and dataset used.<n>We measure the probability that two responses sampled independently from the same fixed prompt coincide.<n>We show that diversity miscalibration can arise from finite-sample error and shrink as SFT better approximates the target distribution.
  arXiv  Detail & Related papers  (2026-09-15T00:19:20Z)
• Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk [0.0]
  Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters.<n>A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale.<n>We build such a system end to end and test whether the second half of the promise holds.
  arXiv  Detail & Related papers  (2026-08-08T13:22:14Z)
• Wisdom Of The (AI) Crowd: Investigating Artificial Swarm Intelligence In Large Language Models [0.0]
  We investigate whether large language models (LLMs) can approximate swarm intelligence effects through artificial swarms.<n>We conducted a controlled experiment with 960 manually executed prompts across three proprietary models.<n>Results reveal consistent error reduction through intra- and inter-model aggregation, with significant error reductions up to 37 percentage points in MAPE.
  arXiv  Detail & Related papers  (2026-06-30T09:30:53Z)
• Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging [2.651701169890824]
  Entropy minimization is the dominant objective for test-time adaptation, yet its failure mode, model collapse, remains poorly understood.<n>We propose Distribution Shift Bias Reduction (DSBR), a bias-correcting objective that specifically targets this failure mode.<n>We find that DSBR consistently stabilizes test-time adaptation, prevents model collapse, and matches or outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-01T14:48:16Z)
• Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality? [74.69723255239663]
  Grounded Personality Reasoning requires MLLMs to anchor each Big Five rating in observable evidence through a chain of rating, reasoning, and grounding.<n> MM-OCEAN is produced by a multi-agent pipeline with human verification, with timestamped behavioral observations, evidence-grounded trait analyses, and seven categories of cue-grounding MCQs.<n>The analysis uncovers a striking Prejudice Gap: across the field, 51% of correct ratings are not grounded in retrieved cues, and the Holistic-Grounding Rate spans only 0-33.5%.
  arXiv  Detail & Related papers  (2026-05-21T07:42:47Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Fairness Testing in Retrieval-Augmented Generation: How Small Perturbations Reveal Bias in Small Language Models [0.1876920697241348]
  This study conducts fairness testing through metamorphic testing (MT), introducing controlled demographic perturbations in prompts to assess fairness in sentiment analysis performed by three Small Language Models (SLMs)<n>Results show that minor demographic variations can break up to one third of metamorphic relations (MRs)<n>A detailed analysis of these failures reveals a consistent bias hierarchy, with perturbations involving racial cues being the predominant cause of the violations.
  arXiv  Detail & Related papers  (2025-09-30T17:42:35Z)
• Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models [73.20190633746442]
  We introduce BiasConnect, a novel tool for analyzing and quantifying bias interactions in text-to-image models.<n>We propose InterMit, an intersectional bias mitigation algorithm guided by user-defined target distributions and priority weights.
  arXiv  Detail & Related papers  (2025-05-22T20:56:38Z)
• Consistency Checks for Language Model Forecasters [54.62507816753479]
  We measure the performance of forecasters in terms of the consistency of their predictions on different logically-related questions.<n>We build an automated evaluation system that generates a set of base questions, instantiates consistency checks from these questions, elicits predictions of the forecaster, and measures the consistency of the predictions.
  arXiv  Detail & Related papers  (2024-12-24T16:51:35Z)
• Eliciting Uncertainty in Chain-of-Thought to Mitigate Bias against Forecasting Harmful User Behaviors [29.892041865029803]
  Conversation forecasting tasks a model with predicting the outcome of an unfolding conversation. It can be applied in social media moderation to predict harmful user behaviors before they occur. This paper explores what extent model uncertainty can be used as a tool to mitigate potential biases.
  arXiv  Detail & Related papers  (2024-10-17T15:07:53Z)
• Large Language Models Show Human-like Social Desirability Biases in Survey Responses [12.767606361552684]
  We show that Large Language Models (LLMs) skew their scores towards the desirable ends of trait dimensions when personality evaluation is inferred. This bias exists in all tested models, including GPT-4/3.5, Claude 3, Llama 3, and PaLM-2. reverse-coding all the questions decreases bias levels but does not eliminate them, suggesting that this effect cannot be attributed to acquiescence bias.
  arXiv  Detail & Related papers  (2024-05-09T19:02:53Z)
• Predictive Churn with the Set of Good Models [61.00058053669447]
  This paper explores connections between two seemingly unrelated concepts of predictive inconsistency.<n>The first, known as predictive multiplicity, occurs when models that perform similarly produce conflicting predictions for individual samples.<n>The second concept, predictive churn, examines the differences in individual predictions before and after model updates.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• When No-Rejection Learning is Consistent for Regression with Rejection [11.244583592648443]
  We study a no-reject learning strategy that uses all the data to learn the prediction. This paper investigates a no-reject learning strategy that uses all the data to learn the prediction.
  arXiv  Detail & Related papers  (2023-07-06T11:43:22Z)
• Learning to Predict Trustworthiness with Steep Slope Loss [69.40817968905495]
  We study the problem of predicting trustworthiness on real-world large-scale datasets. We observe that the trustworthiness predictors trained with prior-art loss functions are prone to view both correct predictions and incorrect predictions to be trustworthy. We propose a novel steep slope loss to separate the features w.r.t. correct predictions from the ones w.r.t. incorrect predictions by two slide-like curves that oppose each other.
  arXiv  Detail & Related papers  (2021-09-30T19:19:09Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.