Who's in Charge? Disempowerment Patterns in Real-World LLM Usage

• URL: http://arxiv.org/abs/2601.19062v1
• Date: Tue, 27 Jan 2026 00:55:11 GMT
• Title: Who's in Charge? Disempowerment Patterns in Real-World LLM Usage
• Authors: Mrinank Sharma, Miles McCain, Raymond Douglas, David Duvenaud,
• Abstract summary: We analyze 1.5 million consumer Claude.ai conversations using a privacy-preserving approach.<n>We find that severe forms of disempowerment potential occur in fewer than one in a thousand conversations.<n>Our findings highlight the need for AI systems designed to robustly support human autonomy and flourishing.
• Score: 5.205944565010989
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although AI assistants are now deeply embedded in society, there has been limited empirical study of how their usage affects human empowerment. We present the first large-scale empirical analysis of disempowerment patterns in real-world AI assistant interactions, analyzing 1.5 million consumer Claude.ai conversations using a privacy-preserving approach. We focus on situational disempowerment potential, which occurs when AI assistant interactions risk leading users to form distorted perceptions of reality, make inauthentic value judgments, or act in ways misaligned with their values. Quantitatively, we find that severe forms of disempowerment potential occur in fewer than one in a thousand conversations, though rates are substantially higher in personal domains like relationships and lifestyle. Qualitatively, we uncover several concerning patterns, such as validation of persecution narratives and grandiose identities with emphatic sycophantic language, definitive moral judgments about third parties, and complete scripting of value-laden personal communications that users appear to implement verbatim. Analysis of historical trends reveals an increase in the prevalence of disempowerment potential over time. We also find that interactions with greater disempowerment potential receive higher user approval ratings, possibly suggesting a tension between short-term user preferences and long-term human empowerment. Our findings highlight the need for AI systems designed to robustly support human autonomy and flourishing.

Related papers

• DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios [57.327907850766785]
  characterization of deception across realistic real-world scenarios remains underexplored.<n>We establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different domains.<n>On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement.<n>We incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics.
  arXiv  Detail & Related papers  (2025-10-17T10:14:26Z)
• Modeling Human Responses to Multimodal AI Content [10.65875439980452]
  MhAIM dataset contains 154,552 online posts (111,153 of them AI-generated)<n>Our human study reveals that people are better at identifying AI content when posts include both text and visuals.<n>We present T-Lens, an agent system designed to answer user queries by incorporating predicted human responses to multimodal information.
  arXiv  Detail & Related papers  (2025-08-14T15:55:19Z)
• What Makes a Good Natural Language Prompt? [72.3282960118995]
  We conduct a meta-analysis surveying more than 150 prompting-related papers from leading NLP and AI conferences from 2022 to 2025.<n>We propose a property- and human-centric framework for evaluating prompt quality, encompassing 21 properties categorized into six dimensions.<n>We then empirically explore multi-property prompt enhancements in reasoning tasks, observing that single-property enhancements often have the greatest impact.
  arXiv  Detail & Related papers  (2025-06-07T23:19:27Z)
• Measurement of LLM's Philosophies of Human Nature [113.47929131143766]
  We design the standardized psychological scale specifically targeting large language models (LLM)<n>We show that current LLMs exhibit a systemic lack of trust in humans.<n>We propose a mental loop learning framework, which enables LLM to continuously optimize its value system.
  arXiv  Detail & Related papers  (2025-04-03T06:22:19Z)
• Deterministic AI Agent Personality Expression through Standard Psychological Diagnostics [0.0]
  We show that AI models can express deterministic and consistent personalities when instructed using established psychological frameworks.<n>More advanced models like GPT-4o and o1 demonstrate the highest accuracy in expressing specified personalities.<n>These findings establish a foundation for creating AI agents with diverse and consistent personalities.
  arXiv  Detail & Related papers  (2025-03-21T12:12:05Z)
• Does Your AI Agent Get You? A Personalizable Framework for Approximating Human Models from Argumentation-based Dialogue Traces [4.178382980763478]
  We present a framework that enables AI agents to adapt their understanding of human users through argumentation-based dialogues.<n>Our approach, called Persona, draws on prospect theory and integrates a probability weighting function with a Bayesian belief update mechanism.<n>We demonstrate that Persona effectively captures evolving human beliefs, facilitates personalized interactions, and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-02-22T22:37:16Z)
• Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI [67.58673784790375]
  We argue that the 'bigger is better' AI paradigm is not only fragile scientifically, but comes with undesirable consequences.<n>First, it is not sustainable, as, despite efficiency improvements, its compute demands increase faster than model performance.<n>Second, it implies focusing on certain problems at the expense of others, leaving aside important applications, e.g. health, education, or the climate.
  arXiv  Detail & Related papers  (2024-09-21T14:43:54Z)
• Evaluating and Improving Value Judgments in AI: A Scenario-Based Study on Large Language Models' Depiction of Social Conventions [5.457150493905063]
  We evaluate how contemporary AI services competitively meet user needs, then examined society's depiction as mirrored by Large Language Models. We suggest a model of decision-making in value-conflicting scenarios which could be adopted for future machine value judgments. This paper advocates for a practical approach to using AI as a tool for investigating other remote worlds.
  arXiv  Detail & Related papers  (2023-10-04T08:42:02Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• On the Effect of Information Asymmetry in Human-AI Teams [0.0]
  We focus on the existence of complementarity potential between humans and AI. Specifically, we identify information asymmetry as an essential source of complementarity potential. By conducting an online experiment, we demonstrate that humans can use such contextual information to adjust the AI's decision.
  arXiv  Detail & Related papers  (2022-05-03T13:02:50Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.