Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents

• URL: http://arxiv.org/abs/2605.08721v1
• Date: Sat, 09 May 2026 06:10:32 GMT
• Title: Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents
• Abstract summary: We propose Dual-scale Evolutionary Policy Training (DEPT) for social language games.<n> DEPT introduces a time-scaled evolutionary perception mechanism that detects impasse.<n>Our method effectively restores gradient signals and enforces sustained strategic exploration.
• Score: 38.60685324043556
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective for closed-ended tasks, extending it to open-ended social language games via self-play reveals a critical issue: evolution impasse. Due to the vast strategy space, language agents frequently converge to homogenized behaviors, leading to deterministic match outcomes that eliminate the gradient signals necessary for policy evolution. To tackle this issue, we propose Dual-scale Evolutionary Policy Training (DEPT) for social language games. DEPT introduces a time-scaled evolutionary perception mechanism that detects impasse by quantifying dual-scale value baseline divergence alongside match entropy. Upon perceiving the collapse, it then activates asymmetric advantage reshaping to dynamically modulate the optimization landscape for intervention. Thus, our method effectively restores gradient signals and enforces sustained strategic exploration. Extensive experiments on multiple social language games demonstrate that DEPT outperforms strong baselines, avoiding policy degeneration and driving the continuous evolution of social language agents.

Related papers

• SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design [52.69818441707104]
  Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue.<n>Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards.<n>We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges.
  arXiv  Detail & Related papers  (2026-09-09T06:06:34Z)
• Self-Improving Large Language Models via Progressive Experience Evolution [52.03479747358687]
  We propose textbfSPEE (textbfSelf-textbfProgressive textbfExperience textbfEvolution), a unified post-training framework that sequentially performs explicit experience evolution followed by implicit policy optimization.<n> Experiments on five mathematical reasoning benchmarks demonstrate that SPEE consistently outperforms both test-time and training-time self-evolution baselines across three model scales.
  arXiv  Detail & Related papers  (2026-08-03T12:27:32Z)
• From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Training [55.744284393275926]
  Reinforcement learning (RL) has become a widely adopted technique for improving large language models (LLMs) on complex tasks.<n>Existing RL methods still face challenges in training agents with longer-horizon interactions.<n>We introduce a novel policy gradient method, Hindsight Policy Optimization (HPO), that projects both the current policy distribution and the hindsight distribution into an intent space.
  arXiv  Detail & Related papers  (2026-06-28T08:35:47Z)
• Self-Evolving Spatial Reasoning in Vision Language Models via Geometric Logic Consistency [53.28076739140119]
  We propose a framework that enforces logical consistency in Vision-Language Models (VLMs) through geometric and linguistic duality operations.<n>SAGE is model-agnostic, data-efficient compared to prior GRPO methods, and can be applied as a lightweight post-training stage to any existing VLM.<n> Experiments on video and spatial reasoning benchmarks demonstrate consistent improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-05-18T10:05:21Z)
• ALSO: Adversarial Online Strategy Optimization for Social Agents [63.84788516416339]
  We propose textbfALSO (textbfAdrial ontextbfLine textbfStrategy textbfOptimization) as the first framework for online strategy optimization in social simulation.<n> ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms.<n>Also introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation.
  arXiv  Detail & Related papers  (2026-05-15T09:25:15Z)
• COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game [15.550671286269209]
  COvolve is a co-evolutionary framework for building continually improving agents.<n>It generates both environments and agent policies, expressed as executable Python code.<n>Our results demonstrate the potential of LLM-driven co-evolution to achieve open-ended learning.
  arXiv  Detail & Related papers  (2026-03-30T12:56:54Z)
• RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback [54.39884046754265]
  RetroAgent is an online RL framework that enables agents to master complex interactive environments.<n>Our experiments show that RetroAgent achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2026-03-09T16:23:33Z)
• Position: Agentic Evolution is the Path to Evolving LLMs [56.733933092220845]
  We argue that addressing this limitation requires a new scaling axis-evolution.<n>Existing deployment-time adaptation methods lack the strategic agency needed to diagnose failures and produce durable improvements.
  arXiv  Detail & Related papers  (2026-01-30T22:15:58Z)
• Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learning [41.90621652673528]
  We propose SPEAR, a curriculum-based self-imitation learning (SIL) recipe for training agentic LLMs.<n>Specifically, our approach incorporates a curriculum to manage the exploration process, utilizing intrinsic rewards to foster skill-level exploration.<n>To further stabilize training, we recalibrate the advantages of experiences in the replay buffer to address the potential policy drift.
  arXiv  Detail & Related papers  (2025-09-26T17:20:38Z)
• Ghost Policies: A New Paradigm for Understanding and Learning from Failure in Deep Reinforcement Learning [0.9790236766474201]
  Arvolution is a novel Augmented Reality (AR) framework.<n>It renders an agent's historical failed policy trajectories as semi-transparent ghosts''<n>We propose a paradigm shift, transforming DRL agent failures into invaluable, actionable learning resources.
  arXiv  Detail & Related papers  (2025-06-14T05:56:42Z)
• Evolution-based Region Adversarial Prompt Learning for Robustness Enhancement in Vision-Language Models [52.8949080772873]
  We propose an evolution-based region adversarial prompt tuning method called ER-APT.<n>In each training iteration, we first generate AEs using traditional gradient-based methods.<n> Subsequently, a genetic evolution mechanism incorporating selection, mutation, and crossover is applied to optimize the AEs.<n>The final evolved AEs are used for prompt tuning, achieving region-based adversarial optimization instead of conventional single-point adversarial prompt tuning.
  arXiv  Detail & Related papers  (2025-03-17T07:08:47Z)
• Simulation of Language Evolution under Regulated Social Media Platforms: A Synergistic Approach of Large Language Models and Genetic Algorithms [6.550725258692423]
  Social media platforms frequently impose restrictive policies to moderate user content, prompting the emergence of creative evasion language strategies.<n>This paper presents a multi-agent framework based on Large Language Models (LLMs) to simulate the iterative evolution of language strategies under regulatory constraints.
  arXiv  Detail & Related papers  (2025-02-26T14:59:27Z)
• Evolution of Societies via Reinforcement Learning [10.689403855269704]
  We propose a methodology that enables simulating populations of Reinforcement Learning agents at evolutionary scale.<n>We derive a fast, parallelizable implementation of Policy Gradient (PG) and Opponent-Learning Awareness (LOLA)<n>We demonstrate our approach by simulating the evolution of very large populations made of heterogeneous co-learning agents, under both naive and advanced learning strategies.
  arXiv  Detail & Related papers  (2024-10-22T22:49:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.