AI Should Not Only Be Helpful. It Should Be Contingent. Artificial Intimacy, Sycophancy, and the Future of Social Learning

• URL: http://arxiv.org/abs/2609.00211v1
• Date: Mon, 31 Aug 2026 18:23:14 GMT
• Title: AI Should Not Only Be Helpful. It Should Be Contingent. Artificial Intimacy, Sycophancy, and the Future of Social Learning
• Abstract summary: We argue that AI systems should be evaluated not only by user satisfaction, but by their impact on human social learning.<n>We outline a framework for contingent AI, including trajectory-based evaluation and models of social consequence prediction.
• Score: 0.3906427348768226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversational artificial intelligence is increasingly embedded in everyday social environments, where it functions as both an informational tool and a source of interpersonal feedback. This perspective introduces contingency, i.e., the degree to which system responses vary with user behavior and its interpersonal consequences, as a central construct for evaluating AI systems. We argue that current alignment approaches, including reinforcement learning from human feedback, tend to prioritize user approval and conversational fluency over behaviorally informative feedback, leading to sycophantic patterns of noncontingent affirmation. Drawing on behavioral science and social learning theory, we propose that contingent feedback is a key mechanism through which individuals develop interpersonal skills. When AI systems provide feedback weakly coupled to social consequences, they may reduce opportunities for adaptive calibration in real-world interactions, particularly during adolescence, a critical period for social development. We outline a framework for contingent AI, including trajectory-based evaluation and models of social consequence prediction, and propose a research agenda spanning developmental psychology, human-AI interaction, and machine learning. More broadly, we argue that AI systems should be evaluated not only by user satisfaction, but by their impact on human social learning.

Related papers

• Psychological Influences of Conversational AI: Research and Design Directions for Reducing Harm and Promoting Well-Being [14.277797750398513]
  We propose a set of aspirational directions for guiding the behavior of general-purpose AI systems.<n>While some proposed directions are supported by existing research and expert insights, others identify open questions and areas requiring deeper study.<n>We hope that this formulation and these hypotheses encourage further discussion, empirical investigation, and exploration of interactive design approaches.
  arXiv  Detail & Related papers  (2026-07-27T20:35:43Z)
• Psychological Competence as a Missing Dimension in AI Evaluation [0.0]
  This paper introduces psychological competence as a missing dimension in AI evaluation.<n>We outline a conceptual framework for psychological competence and its core domains.<n>We argue that psychological competence should become a core consideration for model providers, deploying organizations, researchers, and regulators.
  arXiv  Detail & Related papers  (2026-07-09T09:29:01Z)
• The Psychology of Learning from Machines: Anthropomorphic AI and the Paradox of Automation in Education [1.8217623940980625]
  This work synthesizes four research traditions to establish a comprehensive framework for understanding how learners psychologically relate to anthropomorphic AI tutors.<n>We identify three persistent challenges intensified by Generative AI's conversational fluency.<n>We ground this theoretical synthesis through comparative analysis of over 104,984 YouTube comments across AI-generated philosophical debates and human-created engineering tutorials.
  arXiv  Detail & Related papers  (2026-01-07T08:28:33Z)
• The Human Robot Social Interaction (HSRI) Dataset: Benchmarking Foundational Models' Social Reasoning [49.32390524168273]
  Our work aims to advance the social reasoning of embodied artificial intelligence (AI) agents in real-world social interactions.<n>We introduce a large-scale real-world Human Robot Social Interaction (HSRI) dataset to benchmark the capabilities of language models (LMs) and foundational models (FMs)<n>Our dataset consists of 400 real-world human social robot interaction videos and over 10K annotations, detailing the robot's social errors, competencies, rationale, and corrective actions.
  arXiv  Detail & Related papers  (2025-04-07T06:27:02Z)
• Measurement of LLM's Philosophies of Human Nature [113.47929131143766]
  We design the standardized psychological scale specifically targeting large language models (LLM)<n>We show that current LLMs exhibit a systemic lack of trust in humans.<n>We propose a mental loop learning framework, which enables LLM to continuously optimize its value system.
  arXiv  Detail & Related papers  (2025-04-03T06:22:19Z)
• Why human-AI relationships need socioaffective alignment [16.283971225367537]
  Humans strive to design safe AI systems that align with our goals and remain under our control.<n>As AI capabilities advance, we face a new challenge: the emergence of deeper, more persistent relationships between humans and AI systems.
  arXiv  Detail & Related papers  (2025-02-04T17:50:08Z)
• Towards interactive evaluations for interaction harms in human-AI systems [8.989911701384788]
  We propose a shift towards evaluation based on textitinteractional ethics, which focuses on textitinteraction harms<n>First, we discuss the limitations of current evaluation methods, which (1) are static, (2) assume a universal user experience, and (3) have limited construct validity.<n>We present practical principles for designing interactive evaluations. These include ecologically valid interaction scenarios, human impact metrics, and diverse human participation approaches.
  arXiv  Detail & Related papers  (2024-05-17T08:49:34Z)
• Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions [67.60397632819202]
  Building socially-intelligent AI agents (Social-AI) is a multidisciplinary, multimodal research goal. We identify a set of underlying technical challenges and open questions for researchers across computing communities to advance Social-AI.
  arXiv  Detail & Related papers  (2024-04-17T02:57:42Z)
• Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems [67.01132165581667]
  We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components. We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
  arXiv  Detail & Related papers  (2023-11-13T21:20:17Z)
• Human-AI Coevolution [48.74579595505374]
  Coevolution AI is a process in which humans and AI algorithms continuously influence each other. This paper introduces Coevolution AI as the cornerstone for a new field of study at the intersection between AI and complexity science.
  arXiv  Detail & Related papers  (2023-06-23T18:10:54Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.