Modeling AI Overreliance as a Complex Adaptive System

• URL: http://arxiv.org/abs/2608.19616v1
• Date: Thu, 20 Aug 2026 03:58:39 GMT
• Title: Modeling AI Overreliance as a Complex Adaptive System
• Abstract summary: We study whether people rely on AI assistance appropriately when it is right and checking it when it is not.<n>We model it as a population process: agents repeatedly solve a task alone, accept an AI answer, or verify it.<n>The results frame AI reliance as a computational social dynamics problem.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Whether AI assistance helps or harms a population depends less on the model's accuracy than on whether people rely on it appropriately trusting it when it is right and checking it when it is not. Yet reliance is usually studied one user at a time. We model it as a population process: agents repeatedly solve a task alone, accept an AI answer, or verify it, updating a Bayesian belief about AI quality and, when networked, learning from peers. Four results form one story. The environment sets the baseline: task difficulty and AI quality fix both overreliance and calibration regret. Social learning creates consensus, not overreliance: a mean-preservation theorem, confirmed by a 2*2 topology*tagging design, shows connectivity moves the aggregate only when influence transmits beliefs. Social proof turns reliance into a feedback cascade: visible unverified use suppresses verification and tips the population into collective overreliance. Feedback design can prevent collapse: making verification visible or dampening social proof reverses it. Together, the results frame AI reliance as a computational social dynamics problem, where individual learning, peer observation, and feedback exposure jointly shape whether a population remains calibrated.

Related papers

• Harnessing Image Question Dependence for Better VLM Test-time Reinforcement Learning [66.13367818892503]
  We propose TTIQ, a test-time reinforcement learning framework that harnesses image-question dependence for better vlm adaptation.<n> TTIQ teacher-forces each sampled response under the original image-question pair and its image- and question-ablated variants.<n>It combines image and question dependence with confidence to construct a response-level reward that favors jointly grounded responses.
  arXiv  Detail & Related papers  (2026-09-09T22:54:15Z)
• MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents [74.83535434786145]
  MNIST-PRO is a benchmark that isolates agentic perception by converting MNIST digit recognition into a sequential, glimpse-based search task with lookback constraints.<n>We evaluate ten multimodal models across four memory representations, including raw visual history, textual states, structured metric grid maps, and a consolidated visual canvas.
  arXiv  Detail & Related papers  (2026-08-31T16:05:39Z)
• One AI Signal, Many Human Judgments: A Bayesian Cascade Analysis of AI-based Credibility Indicators in Online Information Spread [39.0786847687173]
  Social media platforms increasingly use AI-based credibility indicators to help users judge misinformation.<n>We introduce a social-learning lens for this setting by extending the classical Bayesian cascade model with the AI indicator as a shared public signal.<n>We show that AI changes what public history means. Through this view, we show that AI changes what public history means.
  arXiv  Detail & Related papers  (2026-08-31T06:27:21Z)
• A Contextual-Bandit Oversight Game with Two-Sided Informational Asymmetry [0.3384279376065155]
  We study runtime human oversight of an AI agent when private information runs in both directions.<n>This is the kind of asymmetry that arises naturally when an autonomous robot or software agent has inspected a situation its human supervisor cannot directly assess.<n>We introduce a contextual-bandit team game with two-sided asymmetric information and a play/ask/trust/oversee interface.
  arXiv  Detail & Related papers  (2026-06-30T20:30:01Z)
• Warning labels shift perceptions of sycophantic AI, but not its influence [63.79516339973759]
  We test whether warning labels mitigate sycophancy's influence.<n>Labeling the system as sycophantic (...may agree with you and validate you even when you are wrong...'') does shift users' perceptions.<n>But it does not reliably reduce sycophancy's influence on users' self-perceived rightness or their willingness to repair the conflict.
  arXiv  Detail & Related papers  (2026-06-19T11:01:45Z)
• Framing, Judging, Steering: An Assessable Competency Model for Teach-ing Students to Reason With Generative AI [47.027290803102666]
  We propose CoRe-3 (Co-Reasoning), a competency model factoring productive AI use into three assessable skills.<n>We ground the skills in theory, state five testable propositions, and instantiate them in CoReasoningLab.<n>Human-rater agreement and outcomes are next; we release the instrument, data, and protocol.
  arXiv  Detail & Related papers  (2026-06-04T10:25:31Z)
• The Missing Knowledge Layer in AI: A Framework for Stable Human-AI Reasoning [0.0]
  Humans exhibit a similar weakness, often mistaking fluency for reliability.<n>This paper is the first in a five-paper research series on stabilising human-AI reasoning.
  arXiv  Detail & Related papers  (2026-04-16T11:19:00Z)
• Learning to Trust: How Humans Mentally Recalibrate AI Confidence Signals [3.1153758106426603]
  We investigate whether humans can learn to mentally recalibrate AI confidence signals through repeated experience.<n>We present a computational model utilizing a linear-in-log-odds transformation and a Rescorla-Wagner learning rule to explain these dynamics.
  arXiv  Detail & Related papers  (2026-03-23T23:23:25Z)
• The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity? [53.15349353876531]
  As AI becomes more capable, we entrust it with more general and consequential tasks.<n>We operationalize this question using a bias-variance decomposition of the errors made by AI models.<n>As more capable AIs pursue harder tasks, requiring more sequential action and thought, our results predict failures to be accompanied by more incoherent behavior.
  arXiv  Detail & Related papers  (2026-01-30T14:52:03Z)
• Lost Before Translation: Social Information Transmission and Survival in AI-AI Communication [7.593123083236325]
  We study what happens when AI talks to AI.<n>In five studies tracking content through AI transmission chains, we find three consistent patterns.<n>We show that the properties that make AI-mediated content appear authoritative may systematically erode the cognitive and affective diversity on which informed judgment depends.
  arXiv  Detail & Related papers  (2026-01-21T17:18:46Z)
• Self-Harmony: Learning to Harmonize Self-Supervision and Self-Play in Test-Time Reinforcement Learning [48.14470449860784]
  Test-time reinforcement learning (TTRL) offers a label-free paradigm for adapting models using only synthetic signals at inference.<n>We introduce Self-Harmony, a framework built on a simple intuition.
  arXiv  Detail & Related papers  (2025-11-03T03:34:34Z)
• Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards [67.86091419220816]
  Large Language Models (LLMs) show great promise in complex reasoning.<n>A prevalent issue is superficial self-reflection'', where models fail to robustly verify their own outputs.<n>We introduce RISE (Reinforcing Reasoning with Self-Verification), a novel online RL framework designed to tackle this.
  arXiv  Detail & Related papers  (2025-05-19T17:59:31Z)
• A Diachronic Perspective on User Trust in AI under Uncertainty [52.44939679369428]
  Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. We study the evolution of user trust in response to trust-eroding events using a betting game.
  arXiv  Detail & Related papers  (2023-10-20T14:41:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.