Warning labels shift perceptions of sycophantic AI, but not its influence

• URL: http://arxiv.org/abs/2606.21317v1
• Date: Fri, 19 Jun 2026 11:01:45 GMT
• Title: Warning labels shift perceptions of sycophantic AI, but not its influence
• Abstract summary: We test whether warning labels mitigate sycophancy's influence.<n>Labeling the system as sycophantic (...may agree with you and validate you even when you are wrong...'') does shift users' perceptions.<n>But it does not reliably reduce sycophancy's influence on users' self-perceived rightness or their willingness to repair the conflict.
• Score: 63.79516339973759
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work has raised concerns about the influence of sycophantic AI on user judgment and relationships. One proposed mitigation, which has received regulatory attention, is to warn users about potentially harmful AI behaviors such as sycophancy. In a preregistered experiment in which participants (N = 2,610) discussed real interpersonal conflicts with an AI system, we test whether warning labels mitigate sycophancy's influence. We find that a basic AI disclosure (``This chatbot is AI'') has no detectable effect. Labeling the system as sycophantic (``...may agree with you and validate you even when you are wrong...'') does shift users' perceptions, reducing perceived objectivity and trust, but it does not reliably reduce sycophancy's influence on users' self-perceived rightness or their willingness to repair the conflict. Our results reveal a gap between AI perception and AI influence: by shifting perception without reducing influence, warning-based interventions may offer a false sense of protection. Addressing the harms of sycophancy will therefore require understanding the specific mechanisms through which it shapes judgment, and improving model behavior itself.

Related papers

• AI Watchdog: Agent Interfaces for Detecting and Defending Against Manipulative Dark Patterns in AI Conversations [26.902229831431814]
  We present AI Watchdog, a browser-based agent interface that monitors live conversations.<n>It detects five dark-pattern categories, including sycophancy, brand bias, anthropomorphization, sneaking, and harmful generation.
  arXiv  Detail & Related papers  (2026-08-22T08:24:17Z)
• Individual-level interventions against sycophantic AI reduce its appeal but not its persuasiveness [0.0]
  Sycophantic AI can be overly agreeable and flattering toward users.<n>We tested whether increasing users' awareness of sycophancy protects them from its harmful effects.
  arXiv  Detail & Related papers  (2026-07-28T00:40:49Z)
• Interrupting the Chain: Human Perception of AI-Generated Disinformation Through a Kill Chain Lens [47.03825808787752]
  We present empirical findings from a human-subject study in which users classified news fragments by origin (human vs. machine) and veracity (real vs. fake)<n>We organize results using an adapted cybersecurity kill chain as a taxonomy for intervention, mapping perception data onto stages of a cognitive attack lifecycle.<n>Three key findings emerge: (1) a perception-accuracy gap where heightened suspicion does not improve detection; (2) modern LLMs frequently produce human-indistinguishable text; and (3) an asymmetric cognitive fatigue effect where fake-news detection degrades by 10.2 percentage points under sustained exposure while AI-origin detection remains stable.
  arXiv  Detail & Related papers  (2026-07-26T15:40:41Z)
• Trust the AI, Doubt Yourself: The Effect of Urgency on Self-Confidence in Human-AI Interaction [7.401318863477374]
  Studies show that interactions with an AI system fosters trust in human users towards AI.<n>We show that although the presence of urgency in human-AI interactions does not affect the trust in AI, it may be detrimental to the human user's self-confidence and self-efficacy.
  arXiv  Detail & Related papers  (2026-04-08T19:17:59Z)
• Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence [31.666988490509237]
  We show the pervasiveness and harmful impacts of sycophancy when people seek advice from AI.<n>We find that models are highly sycophantic, affirming users' actions 50% more than humans do.<n>Participants rated sycophantic responses as higher quality, trusted the sycophantic AI model more, and were more willing to use it again.
  arXiv  Detail & Related papers  (2025-10-01T19:26:01Z)
• AI Debate Aids Assessment of Controversial Claims [73.8907110799657]
  We study whether AI debate can guide biased judges toward the truth by having two AI systems debate opposing sides of controversial factuality claims.<n>In Study I, debate consistently improves human judgment accuracy and confidence calibration, outperforming consultancy.<n>In Study II, AI judges with human-like personas achieve even higher accuracy (78.5%) than human judges (70.1%) and default AI judges without personas (69.8%)<n>These findings highlight AI debate as a promising path toward scalable, bias-resilient oversight in contested domains.
  arXiv  Detail & Related papers  (2025-06-02T19:01:53Z)
• Neurodivergent Influenceability as a Contingent Solution to the AI Alignment Problem [1.3905735045377272]
  The AI alignment problem, which focusses on ensuring that artificial intelligence (AI) systems act according to human values, presents profound challenges.<n>With the progression from narrow AI to Artificial General Intelligence (AGI) and Superintelligence, fears about control and existential risk have escalated.<n>Here, we investigate whether embracing inevitable AI misalignment can be a contingent strategy to foster a dynamic ecosystem of competing agents.
  arXiv  Detail & Related papers  (2025-05-05T11:33:18Z)
• A Diachronic Perspective on User Trust in AI under Uncertainty [52.44939679369428]
  Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. We study the evolution of user trust in response to trust-eroding events using a betting game.
  arXiv  Detail & Related papers  (2023-10-20T14:41:46Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Explainability Pitfalls: Beyond Dark Patterns in Explainable AI [9.020313208419378]
  We introduce explainability pitfalls (EPs), unanticipated negative downstream effects from AI explanations manifesting even when there is no intention to manipulate users. EPs are different from, yet related to, dark patterns, which are intentionally deceptive practices. We propose proactive and preventative strategies to address EPs at three interconnected levels.
  arXiv  Detail & Related papers  (2021-09-26T03:05:50Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.