Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding

• URL: http://arxiv.org/abs/2608.05832v1
• Date: Thu, 06 Aug 2026 10:00:50 GMT
• Title: Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding
• Abstract summary: Think-Strategy-Response (TSR) framework decomposes social dialogue into two hierarchical stages: high-level strategic planning and low-level linguistic execution.<n>We introduce Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR), a novel algorithm that dynamically routes rewards.<n> Experiments show that our approach fine-tunes a Qwen2.5-7B agent to surpass the GPT-4o baseline by 7.32% in goal completion success.
• Score: 36.605789019423355
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Large language models (LLMs) excel in structured tasks but struggle with dynamic social interactions, where success requires long-term goal coordination and rapid adaptation. Current methods often apply uniform goal-based rewards to every utterance, overlooking the specificity of objectives at each dialogue turn and failing to account for the rationale of potential strategies. Inspired by the Theory of Planned Behavior, we propose the Think-Strategy-Response (TSR) framework, which decomposes social dialogue into two hierarchical stages: high-level strategic planning and low-level linguistic execution. To optimize TSR, we introduce Linearized Hierarchical Reinforcement Learning with Variance-Gated Rewards (LHRL-VGR), a novel algorithm that dynamically routes rewards - balancing goal completion and strategy adherence - based on the variance of goal achievement scores. Experiments on the SOTOPIA benchmark show that our approach fine-tunes a Qwen2.5-7B agent to surpass the GPT-4o baseline by 7.32% in goal completion success, demonstrating state-of-the-art performance in multi-agent social negotiation tasks.

Related papers

• SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design [52.69818441707104]
  Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue.<n>Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards.<n>We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges.
  arXiv  Detail & Related papers  (2026-09-09T06:06:34Z)
• ToSCA: Leveraging Hierarchical Reinforcement Learning on Temporal and Strategic Abstractions of Conversational Agents [9.356868740164975]
  We propose a two-level hierarchical reinforcement learning framework for conversational agents.<n>The token-level response decoding is conditioned on the utterance-level action.<n>Our method outperforms versatile baselines in strategy determination and response quality.
  arXiv  Detail & Related papers  (2026-08-22T14:15:56Z)
• ALSO: Adversarial Online Strategy Optimization for Social Agents [63.84788516416339]
  We propose textbfALSO (textbfAdrial ontextbfLine textbfStrategy textbfOptimization) as the first framework for online strategy optimization in social simulation.<n> ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms.<n>Also introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation.
  arXiv  Detail & Related papers  (2026-05-15T09:25:15Z)
• Reinforcement Learning with Anticipation: A Hierarchical Approach for Long-Horizon Tasks [3.79187263097166]
  Solving long-horizon goal-conditioned tasks remains a significant challenge in reinforcement learning.<n>We introduce Reinforcement Learning with Anticipation (RLA), a principled and potentially scalable framework designed to address these limitations.<n>Key feature of RLA is the training of the anticipation model, which is guided by a principle of value geometric consistency.
  arXiv  Detail & Related papers  (2025-09-06T00:10:15Z)
• Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning [56.496001894673235]
  Reinforcement Learning (RL) has proven highly effective at enhancing the complex reasoning abilities of Large Language Models (LLMs)<n>Our analysis reveals that puzzling phenomena like aha moments", length-scaling'' and entropy dynamics are not disparate occurrences but hallmarks of an emergent reasoning hierarchy.
  arXiv  Detail & Related papers  (2025-09-03T18:52:49Z)
• SOTOPIA-$Ω$: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agents [16.320531397370008]
  We propose a framework for enhancing the social capabilities of language agents.<n>We introduce the concept of Social Instruction Following (S-IF) and propose two new S-IF evaluation metrics.<n>We demonstrate that several 7B models trained on high-quality corpus not only significantly surpass the expert agent (GPT-4) in achieving social goals.
  arXiv  Detail & Related papers  (2025-02-21T15:40:37Z)
• EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning [69.55982246413046]
  We propose explicit policy optimization (EPO) for strategic reasoning.<n>We train the strategic reasoning model via multi-turn reinforcement learning (RL),utilizing process rewards and iterative self-play.<n>Our findings reveal various collaborative reasoning mechanisms emergent in EPO and its effectiveness in generating novel strategies.
  arXiv  Detail & Related papers  (2025-02-18T03:15:55Z)
• Retrieval-Augmented Hierarchical in-Context Reinforcement Learning and Hindsight Modular Reflections for Task Planning with LLMs [8.55917897789612]
  We propose Retrieval-Augmented in-context reinforcement Learning (RAHL) for large language models. RAHL decomposes complex tasks into sub-tasks using an LLM-based high-level policy. We show that RAHL can achieve an improvement in performance in 9%, 42%, and 10% in 5 episodes of execution in strong baselines.
  arXiv  Detail & Related papers  (2024-08-12T22:40:01Z)
• K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning [76.3114831562989]
  It requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments. We propose a novel framework: "K-Level Reasoning with Large Language Models (K-R)"
  arXiv  Detail & Related papers  (2024-02-02T16:07:05Z)
• Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph [21.260858893505183]
  We present a novel two-layer hierarchical learning approach equipped with a Goals Graph (GRG) Our GRG captures the underlying relations of all goals in the goal space through a Dirichlet-categorical that process. Our experimental results show that our approach exhibits superior generalization on both unseen environments and new goals.
  arXiv  Detail & Related papers  (2021-03-01T23:21:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.