SOTOPIA-$Ω$: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agents

• URL: http://arxiv.org/abs/2502.15538v3
• Date: Thu, 29 May 2025 08:54:31 GMT
• Title: SOTOPIA-$Ω$: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agents
• Abstract summary: We propose a framework for enhancing the social capabilities of language agents.<n>We introduce the concept of Social Instruction Following (S-IF) and propose two new S-IF evaluation metrics.<n>We demonstrate that several 7B models trained on high-quality corpus not only significantly surpass the expert agent (GPT-4) in achieving social goals.
• Score: 16.320531397370008
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the abundance of prior social strategies possessed by humans, there remains a paucity of research dedicated to their transfer and integration into social agents. Our proposed SOTOPIA-$\Omega$ framework aims to address and bridge this gap, with a particular focus on enhancing the social capabilities of language agents. This framework dynamically injects multi-step reasoning strategies inspired by negotiation theory and two simple direct strategies into expert agents, thereby automating the construction of a high-quality social dialogue training corpus. Additionally, we introduce the concept of Social Instruction Following (S-IF) and propose two new S-IF evaluation metrics that complement social capability. We demonstrate that several 7B models trained on high-quality corpus not only significantly surpass the expert agent (GPT-4) in achieving social goals but also enhance S-IF performance. Analysis and variant experiments validate the advantages of dynamic construction, which can especially break the agent's prolonged deadlock.

Related papers

• SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design [52.69818441707104]
  Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue.<n>Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards.<n>We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges.
  arXiv  Detail & Related papers  (2026-09-09T06:06:34Z)
• Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding [36.605789019423355]
  Think-Strategy-Response (TSR) framework decomposes social dialogue into two hierarchical stages: high-level strategic planning and low-level linguistic execution.<n>We introduce Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR), a novel algorithm that dynamically routes rewards.<n> Experiments show that our approach fine-tunes a Qwen2.5-7B agent to surpass the GPT-4o baseline by 7.32% in goal completion success.
  arXiv  Detail & Related papers  (2026-08-06T10:00:50Z)
• ALSO: Adversarial Online Strategy Optimization for Social Agents [63.84788516416339]
  We propose textbfALSO (textbfAdrial ontextbfLine textbfStrategy textbfOptimization) as the first framework for online strategy optimization in social simulation.<n> ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms.<n>Also introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation.
  arXiv  Detail & Related papers  (2026-05-15T09:25:15Z)
• SODE: Analyzing Social Dynamics in LLM Agents [17.885513252130373]
  We introduce SODE (Social Dynamics Evaluation), a framework that evaluates AI agents across three evolutionary dimensions.<n>Applying SODE reveals systematic divergences.<n>We demonstrate that a "long-horizon framing" can unlock reciprocal capabilities in reasoning models.
  arXiv  Detail & Related papers  (2026-05-06T14:50:07Z)
• SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution [82.31558282651811]
  Social intelligence, the ability to navigate complex interpersonal interactions, presents a fundamental challenge for language agents.<n>Existing approaches directly employ language models to distribute episode-level rewards, yielding attributions that are retrospective and lack theoretical grounding.<n>We propose SAVOIR, a novel principled framework grounded in cooperative game theory.
  arXiv  Detail & Related papers  (2026-04-21T02:08:25Z)
• Social-R1: Towards Human-like Social Reasoning in LLMs [74.32494331695837]
  We argue that cultivating human-like social intelligence requires training with challenging cases that resist shortcut solutions.<n>We propose Social-R1, a reinforcement learning framework that aligns model reasoning with human cognition through multi-dimensional rewards.
  arXiv  Detail & Related papers  (2026-03-10T06:26:24Z)
• Simulating Generative Social Agents via Theory-Informed Workflow Design [11.992123170134185]
  We propose a theory-informed framework that provides a systematic design process for social agents.<n>Our framework is grounded in principles from Social Cognition Theory and introduces three key modules: motivation, action planning, and learning.<n>Experiments demonstrate that our theory-driven agents reproduce realistic human behavior patterns under complex conditions.
  arXiv  Detail & Related papers  (2025-08-12T08:14:48Z)
• Sotopia-RL: Reward Design for Social Intelligence [52.59432715228559]
  Sotopia-RL is a novel framework that refines coarse episode-level feedback into utterance-level, multi-dimensional rewards.<n>Experiments in Sotopia, an open-ended social learning environment, demonstrate that Sotopia-RL achieves state-of-the-art social goal completion scores.
  arXiv  Detail & Related papers  (2025-08-05T20:43:42Z)
• LLM Economist: Large Population Models and Mechanism Design in Multi-Agent Generative Simulacra [29.627070781534698]
  We present a novel framework that uses agent-based modeling to design and assess economic policies.<n>At the lower level, bounded rational worker agents choose labor supply to maximize text-based utility functions learned in-context.<n>At the upper level, a planner agent employs in-context reinforcement learning to propose piecewise-linear marginal tax schedules anchored to the current U.S. federal brackets.
  arXiv  Detail & Related papers  (2025-07-21T17:21:14Z)
• EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning [69.55982246413046]
  We propose explicit policy optimization (EPO) for strategic reasoning. EPO provides strategies in open-ended action space and can be plugged into arbitrary LLM agents to motivate goal-directed behavior. Experiments across social and physical domains demonstrate EPO's ability of long-term goal alignment.
  arXiv  Detail & Related papers  (2025-02-18T03:15:55Z)
• AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios [38.878966229688054]
  We introduce AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios. Drawing on Dramaturgical Theory, AgentSense employs a bottom-up approach to create 1,225 diverse social scenarios constructed from extensive scripts. We analyze goals using ERG theory and conduct comprehensive experiments. Our findings highlight that LLMs struggle with goals in complex social scenarios, especially high-level growth needs, and even GPT-4o requires improvement in private information reasoning.
  arXiv  Detail & Related papers  (2024-10-25T07:04:16Z)
• SocialGFs: Learning Social Gradient Fields for Multi-Agent Reinforcement Learning [58.84311336011451]
  We propose a novel gradient-based state representation for multi-agent reinforcement learning. We employ denoising score matching to learn the social gradient fields (SocialGFs) from offline samples. In practice, we integrate SocialGFs into the widely used multi-agent reinforcement learning algorithms, e.g., MAPPO.
  arXiv  Detail & Related papers  (2024-05-03T04:12:19Z)
• SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents [73.35393511272791]
  We propose an interactive learning method, SOTOPIA-$pi$, improving the social intelligence of language agents. This method leverages behavior cloning and self-reinforcement training on filtered social interaction data according to large language model (LLM) ratings.
  arXiv  Detail & Related papers  (2024-03-13T17:17:48Z)
• Emergence of Social Norms in Generative Agent Societies: Principles and Architecture [8.094425852451643]
  We propose a novel architecture, named CRSEC, to empower the emergence of social norms within generative MASs. Our architecture consists of four modules: Creation & Representation, Spreading, Evaluation, and Compliance. Our experiments demonstrate the capability of our architecture to establish social norms and reduce social conflicts within generative MASs.
  arXiv  Detail & Related papers  (2024-03-13T05:08:10Z)
• K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning [76.3114831562989]
  It requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments. We propose a novel framework: "K-Level Reasoning with Large Language Models (K-R)"
  arXiv  Detail & Related papers  (2024-02-02T16:07:05Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)
• Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View [60.80731090755224]
  This paper probes the collaboration mechanisms among contemporary NLP systems by practical experiments with theoretical insights. We fabricate four unique societies' comprised of LLM agents, where each agent is characterized by a specific trait' (easy-going or overconfident) and engages in collaboration with a distinct thinking pattern' (debate or reflection) Our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring social psychology theories.
  arXiv  Detail & Related papers  (2023-10-03T15:05:52Z)
• Towards socially-competent and culturally-adaptive artificial agents Expressive order, interactional disruptions and recovery strategies [0.0]
  The overarching aim of this work is to set a framework to make the artificial agent socially-competent beyond dyadic interaction-interaction. The paper highlights how this level of competence is achieved by focusing on just three dimensions: (i) social capability, (ii) relational role, and (iii) proximity.
  arXiv  Detail & Related papers  (2023-08-06T15:47:56Z)
• Training Socially Aligned Language Models on Simulated Social Interactions [99.39979111807388]
  Social alignment in AI systems aims to ensure that these models behave according to established societal values. Current language models (LMs) are trained to rigidly replicate their training corpus in isolation. This work presents a novel training paradigm that permits LMs to learn from simulated social interactions.
  arXiv  Detail & Related papers  (2023-05-26T14:17:36Z)
• Towards Social Identity in Socio-Cognitive Agents [0.0]
  We propose a socio-cognitive agent model based on the concept of Cognitive Social Frames. Cognitive Social Frames can be built around social groups, and form the basis for social group dynamics mechanisms and construct of Social Identity.
  arXiv  Detail & Related papers  (2020-01-20T15:27:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.