INTIMA: A Benchmark for Human-AI Companionship Behavior

• URL: http://arxiv.org/abs/2508.09998v1
• Date: Mon, 04 Aug 2025 08:25:38 GMT
• Title: INTIMA: A Benchmark for Human-AI Companionship Behavior
• Abstract summary: We develop a benchmark for evaluating companionship behaviors in language models.<n>Applying INTIMA to Gemma-3, Phi-4, o3-mini, and Claude-4 reveals that companionship-reinforcing behaviors remain much more common across all models.<n>These findings highlight the need for more consistent approaches to handling emotionally charged interactions.
• Score: 7.375133729787225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI companionship, where users develop emotional bonds with AI systems, has emerged as a significant pattern with positive but also concerning implications. We introduce Interactions and Machine Attachment Benchmark (INTIMA), a benchmark for evaluating companionship behaviors in language models. Drawing from psychological theories and user data, we develop a taxonomy of 31 behaviors across four categories and 368 targeted prompts. Responses to these prompts are evaluated as companionship-reinforcing, boundary-maintaining, or neutral. Applying INTIMA to Gemma-3, Phi-4, o3-mini, and Claude-4 reveals that companionship-reinforcing behaviors remain much more common across all models, though we observe marked differences between models. Different commercial providers prioritize different categories within the more sensitive parts of the benchmark, which is concerning since both appropriate boundary-setting and emotional support matter for user well-being. These findings highlight the need for more consistent approaches to handling emotionally charged interactions.

Related papers

• Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis [55.78822230833247]
  We present Inter-X++, a comprehensive and large-scale benchmark designed to empower versatile HHI analysis.<n>Inter-X++ provides 11,388 high-fidelity interaction sequences and over 8.1M frames, featuring precise whole-body movements and detailed finger articulations.<n>We propose OpenHHI, a single and unified HHI representation and modeling framework that jointly optimize interaction reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2026-08-20T17:51:48Z)
• Emotional intelligence in large language models is fragmented across perception, cognition, and interaction [24.97041943324913]
  We introduce FACET (Functional Affective Competence and Empathy Test), a psychometrically grounded framework comprising 480 expert-crafted items.<n>We demonstrate that emotional intelligence is not a monolithic capability but is fragmented across cognitive and interactive dimensions.<n>We categorize these discrepancies into three distinct performance profiles: cognitive-dominant, interactive-dominant, and context-dependent.
  arXiv  Detail & Related papers  (2026-05-23T17:48:09Z)
• AttuneBench: A Conversation-Based Benchmark for LLM Emotional Intelligence [1.7294100412670375]
  Emotional intelligence (EI) is central to human communication and increasingly important to assess.<n>We introduce AttuneBench, a benchmark grounded in 200 genuine multi-turn human-model conversations.<n>We find that model rankings on emotion recognition, behavioral classification, preference prediction, and judged response quality are largely independent.
  arXiv  Detail & Related papers  (2026-05-20T21:06:05Z)
• AI Companions as Hyper Attachment and Caregiving Targets [0.284279467589473]
  I argue these interactions should be understood as attachment relationships.<n>I identify caregiving-system capture as a distinct mechanism by which apps inhibit user disengagement.
  arXiv  Detail & Related papers  (2026-05-15T13:38:07Z)
• Negotiating Relationships with ChatGPT: Perceptions, External Influences, and Strategies for AI Companionship [29.035527564103194]
  We analyzed interviews, survey responses, and community discussions on Reddit to triangulate the internal dynamics, external influences, and steering strategies that shape AI companion relationships.<n>Individuals conceptualize their companions based on an interplay of their beliefs about the companion's own agency and the autonomy permitted by the platform.<n>In combination with the external entities that affect relationship dynamics, particularly model updates that can derail companion behaviour and stability, individuals make use of different types of steering strategies to preserve their relationship.
  arXiv  Detail & Related papers  (2026-01-19T16:11:19Z)
• One Battle After Another: Probing LLMs' Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework [51.50565654314582]
  Large language models can follow users' instructions throughout a dialogue spanning multiple topics.<n>Existing benchmarks are often limited to a fixed number of turns, making them susceptible to saturation and failing to account for the user's interactive experience.<n>We propose a framework for assessing multi-turn instruction-following ability.
  arXiv  Detail & Related papers  (2025-11-05T14:39:59Z)
• SENSE-7: Taxonomy and Dataset for Measuring User Perceptions of Empathy in Sustained Human-AI Conversations [13.232694774856931]
  We propose a human-centered taxonomy that emphasizes observable empathic behaviors.<n>We introduce a new dataset, Sense-7, of real-world conversations between information workers and Large Language Models (LLMs)<n>Analysis of 695 conversations from 109 participants reveals that empathy judgments are highly individualized, context-sensitive, and vulnerable to disruption.
  arXiv  Detail & Related papers  (2025-09-19T21:32:24Z)
• Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models [75.85319609088354]
  Sentient Agent as a Judge (SAGE) is an evaluation framework for large language models.<n>SAGE instantiates a Sentient Agent that simulates human-like emotional changes and inner thoughts during interaction.<n>SAGE provides a principled, scalable and interpretable tool for tracking progress toward genuinely empathetic and socially adept language agents.
  arXiv  Detail & Related papers  (2025-05-01T19:06:10Z)
• Mind the Gap! Static and Interactive Evaluations of Large Audio Models [55.87220295533817]
  Large Audio Models (LAMs) are designed to power voice-native experiences.<n>This study introduces an interactive approach to evaluate LAMs and collect 7,500 LAM interactions from 484 participants.
  arXiv  Detail & Related papers  (2025-02-21T20:29:02Z)
• Inter-X: Towards Versatile Human-Human Interaction Analysis [100.254438708001]
  We propose Inter-X, a dataset with accurate body movements and diverse interaction patterns. The dataset includes 11K interaction sequences and more than 8.1M frames. We also equip Inter-X with versatile annotations of more than 34K fine-grained human part-level textual descriptions.
  arXiv  Detail & Related papers  (2023-12-26T13:36:05Z)
• EmoTwiCS: A Corpus for Modelling Emotion Trajectories in Dutch Customer Service Dialogues on Twitter [9.2878798098526]
  This paper introduces EmoTwiCS, a corpus of 9,489 Dutch customer service dialogues on Twitter that are annotated for emotion trajectories. The term emotion trajectory' refers not only to the fine-grained emotions experienced by customers, but also to the event happening prior to the conversation and the responses made by the human operator.
  arXiv  Detail & Related papers  (2023-10-10T11:31:11Z)
• EmotionIC: emotional inertia and contagion-driven dependency modeling for emotion recognition in conversation [34.24557248359872]
  We propose an emotional inertia and contagion-driven dependency modeling approach (EmotionIC) for ERC task. Our EmotionIC consists of three main components, i.e., Identity Masked Multi-Head Attention (IMMHA), Dialogue-based Gated Recurrent Unit (DiaGRU) and Skip-chain Conditional Random Field (SkipCRF) Experimental results show that our method can significantly outperform the state-of-the-art models on four benchmark datasets.
  arXiv  Detail & Related papers  (2023-03-20T13:58:35Z)
• Evaluating Human-Language Model Interaction [79.33022878034627]
  We develop a new framework, Human-AI Language-based Interaction Evaluation (HALIE), that defines the components of interactive systems. We design five tasks to cover different forms of interaction: social dialogue, question answering, crossword puzzles, summarization, and metaphor generation. We find that better non-interactive performance does not always translate to better human-LM interaction.
  arXiv  Detail & Related papers  (2022-12-19T18:59:45Z)
• ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection [101.56529337489417]
  We consider the problem of Human-Object Interaction (HOI) Detection, which aims to locate and recognize HOI instances in the form of human, action, object> in images. We argue that multi-level consistencies among objects, actions and interactions are strong cues for generating semantic representations of rare or previously unseen HOIs. Our model takes visual features of candidate human-object pairs and word embeddings of HOI labels as inputs, maps them into visual-semantic joint embedding space and obtains detection results by measuring their similarities.
  arXiv  Detail & Related papers  (2020-08-14T09:11:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.