SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design

• URL: http://arxiv.org/abs/2609.09764v1
• Date: Wed, 09 Sep 2026 06:06:34 GMT
• Title: SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design
• Abstract summary: Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue.<n>Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards.<n>We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges.
• Score: 52.69818441707104
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue. As language models become autonomous collaborators, it is central to building effective and trustworthy human-AI interaction. Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards, producing short-sighted policies that struggle to manage goal-relationship tensions across multi-turn interactions. We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges. First, we apply multi-turn reinforcement learning using PPO that propagates delayed outcome rewards back to each turn, enabling long-horizon planning. Second, we design six process reward dimensions capturing the goal-relationship trade-off, including goal advancement, relational attunement, contextual coherence, etc. A reward model dynamically generates fine-grained scoring criteria for each dimension, while a stage-aware weight schedule prioritizes relationship-building in early turns, goal advancement mid-way, and balanced closure late. Across multiple social-dialogue benchmarks, SocialRL improves Goal Achievement by an average of 9.2 percentage points over the corresponding Base models. These results demonstrate the effectiveness of SocialRL across synthetic and real social scenes, as well as standard and challenging social scenarios.

Related papers

• Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding [36.605789019423355]
  Think-Strategy-Response (TSR) framework decomposes social dialogue into two hierarchical stages: high-level strategic planning and low-level linguistic execution.<n>We introduce Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR), a novel algorithm that dynamically routes rewards.<n> Experiments show that our approach fine-tunes a Qwen2.5-7B agent to surpass the GPT-4o baseline by 7.32% in goal completion success.
  arXiv  Detail & Related papers  (2026-08-06T10:00:50Z)
• ALSO: Adversarial Online Strategy Optimization for Social Agents [63.84788516416339]
  We propose textbfALSO (textbfAdrial ontextbfLine textbfStrategy textbfOptimization) as the first framework for online strategy optimization in social simulation.<n> ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms.<n>Also introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation.
  arXiv  Detail & Related papers  (2026-05-15T09:25:15Z)
• PhySE: A Psychological Framework for Real-Time AR-LLM Social Engineering Attacks [11.801855344327928]
  Malicious actors use Augmented Reality (AR) glasses to capture a target visual and vocal data.<n>Large Language Model (LLM) analyzes this data to identify the individual and generate a detailed social profile.<n>LLM-powered agents employ social engineering strategies, providing real-time conversation suggestions to gain the target trust.
  arXiv  Detail & Related papers  (2026-04-25T05:31:40Z)
• Social-R1: Towards Human-like Social Reasoning in LLMs [74.32494331695837]
  We argue that cultivating human-like social intelligence requires training with challenging cases that resist shortcut solutions.<n>We propose Social-R1, a reinforcement learning framework that aligns model reasoning with human cognition through multi-dimensional rewards.
  arXiv  Detail & Related papers  (2026-03-10T06:26:24Z)
• From Obstacles to Etiquette: Robot Social Navigation with VLM-Informed Path Selection [57.74400052368147]
  This paper presents a social robot navigation framework that integrates geometric planning with contextual social reasoning.<n>The system first extracts obstacles and human dynamics to generate geometrically feasible candidate paths, then leverages a fine-tuned vision-language model (VLM) to evaluate these paths.<n>Experiments in four social navigation contexts demonstrate that our method achieves the best overall performance with the lowest personal space violation duration, the minimal pedestrian-facing time, and no social zone intrusions.
  arXiv  Detail & Related papers  (2026-02-09T18:46:12Z)
• One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligence [25.89075578734277]
  This paper introduces OMAR: One Model, All Roles, a reinforcement learning framework for AI.<n>OMAR allows a single model to role-play all participants in a conversation simultaneously, learning to achieve long-term goals and complex social norms.<n>We show that trained models develop fine-grained, emergent social intelligence, such as empathy, persuasion, and compromise seeking.
  arXiv  Detail & Related papers  (2026-02-03T05:09:49Z)
• Sotopia-RL: Reward Design for Social Intelligence [52.59432715228559]
  Sotopia-RL is a novel framework that refines coarse episode-level feedback into utterance-level, multi-dimensional rewards.<n>Experiments in Sotopia, an open-ended social learning environment, demonstrate that Sotopia-RL achieves state-of-the-art social goal completion scores.
  arXiv  Detail & Related papers  (2025-08-05T20:43:42Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.