ALSO: Adversarial Online Strategy Optimization for Social Agents

• URL: http://arxiv.org/abs/2605.15768v2
• Date: Tue, 19 May 2026 06:16:22 GMT
• Title: ALSO: Adversarial Online Strategy Optimization for Social Agents
• Abstract summary: We propose textbfALSO (textbfAdrial ontextbfLine textbfStrategy textbfOptimization) as the first framework for online strategy optimization in social simulation.<n> ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms.<n>Also introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation.
• Score: 63.84788516416339
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social simulation provides a compelling testbed for studying social intelligence, where agents interact through multi-turn dialogues under evolving contexts and strategically adapting opponents. Such environments are inherently non-stationary, requiring agents to dynamically adjust their strategies over time. However, most Large Language Model (LLM) based social agents rely on static personas, while existing approaches for enhancing social intelligence, such as offline reinforcement learning or external planners, are ill-suited to these settings, typically assuming stationarity and incurring substantial training overhead. To bridge this gap, we propose \textbf{ALSO} (\textbf{A}dversarial on\textbf{L}ine \textbf{S}trategy \textbf{O}ptimization), the first framework for online strategy optimization in multi-agent social simulation. ALSO advances social adaptation through two key contributions. (1) ALSO formulates multi-turn interaction as an adversarial bandit problem, where combinations of static personas and dynamic strategy instructions are treated as arms, providing a principled solution to non-stationarity without relying on environmental stability assumptions. (2) To predict rewards and generalize sparse feedback in multi-turn dialogues, ALSO introduces a lightweight neural surrogate to predict rewards from interaction histories, enabling sample-efficient exploration and continuous online adaptation. Experiments on the Sotopia benchmark demonstrate that ALSO consistently outperforms static baselines and existing optimization methods in dynamic environments, validating the effectiveness of adversarial online strategy optimization for building robust social agents.

Related papers

• SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design [52.69818441707104]
  Social intelligence enables agents to read social context, infer intent, and adapt over sustained dialogue.<n>Existing reinforcement learning methods optimize single-turn utterances and sparse outcome rewards.<n>We propose SocialRL, a multi-turn reinforcement learning framework addressing both challenges.
  arXiv  Detail & Related papers  (2026-09-09T06:06:34Z)
• CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents [53.765941044015854]
  We propose Character-Centric Group Relative Policy Optimization (CRPO), a framework designed to realign objectives with the role-playing task.<n>CRPO improves character distinctiveness through three mechanisms: decoupling task logic from stylistic rewards to resolve gradient conflicts, dynamically adapting optimization constraints based on character complexity, and utilizing generic responses as negative baselines to prevent the model from reverting to a common distribution.
  arXiv  Detail & Related papers  (2026-05-25T07:15:38Z)
• SODE: Analyzing Social Dynamics in LLM Agents [17.885513252130373]
  We introduce SODE (Social Dynamics Evaluation), a framework that evaluates AI agents across three evolutionary dimensions.<n>Applying SODE reveals systematic divergences.<n>We demonstrate that a "long-horizon framing" can unlock reciprocal capabilities in reasoning models.
  arXiv  Detail & Related papers  (2026-05-06T14:50:07Z)
• PolicySim: An LLM-Based Agent Social Simulation Sandbox for Proactive Policy Optimization [22.490419080772586]
  PolicySim is an LLM-based social simulation sandbox for the proactive assessment and optimization of intervention policies.<n>We show that PolicySim can accurately simulate platform ecosystems at both micro and macro levels.
  arXiv  Detail & Related papers  (2026-03-20T05:20:39Z)
• Game-Theoretic Co-Evolution for LLM-Based Heuristic Discovery [37.96481049421407]
  Large language models (LLMs) have enabled rapid progress in automatic discovery.<n>We propose a game-theoretic framework that reframes discovery as a program level co-evolution between solver and instance generator.
  arXiv  Detail & Related papers  (2026-01-30T12:14:52Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• A Hierarchical Hybrid AI Approach: Integrating Deep Reinforcement Learning and Scripted Agents in Combat Simulations [0.0]
  This paper introduces a novel hierarchical hybrid artificial intelligence (AI) approach that synergizes the reliability and predictability of scripted agents with the dynamic, adaptive learning capabilities of RL.<n>By structuring the AI system hierarchically, the proposed approach aims to utilize scripted agents for routine, tactical-level decisions and RL agents for higher-level, strategic decision-making.
  arXiv  Detail & Related papers  (2025-11-28T23:50:29Z)
• Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization [51.12422886183246]
  Large Language Models (LLMs) have developed rapidly in web services, delivering unprecedented capabilities while amplifying societal risks.<n>Existing works tend to focus on either isolated jailbreak attacks or static defenses, neglecting the dynamic interplay between evolving threats and safeguards in real-world web contexts.<n>We propose ACE-Safety, a novel framework that jointly optimize attack and defense models by seamlessly integrating two key innovative procedures.
  arXiv  Detail & Related papers  (2025-11-24T15:23:41Z)
• Grounded Test-Time Adaptation for LLM Agents [75.62784644919803]
  Large language model (LLM)-based agents struggle to generalize to novel and complex environments.<n>We propose two strategies for adapting LLM agents by leveraging environment-specific information available during deployment.
  arXiv  Detail & Related papers  (2025-11-06T22:24:35Z)
• CyGATE: Game-Theoretic Cyber Attack-Defense Engine for Patch Strategy Optimization [73.13843039509386]
  This paper presents CyGATE, a game-theoretic framework modeling attacker-defender interactions.<n>CyGATE frames cyber conflicts as a partially observable game (POSG) across Cyber Kill Chain stages.<n>The framework's flexible architecture enables extension to multi-agent scenarios.
  arXiv  Detail & Related papers  (2025-08-01T09:53:06Z)
• GGBond: Growing Graph-Based AI-Agent Society for Socially-Aware Recommender Simulation [2.7083394633019973]
  We propose a high-fidelity social simulation platform to realistically simulate user behavior evolution under recommendation interventions.<n>The system comprises a population of Sim-User Agents equipped with a five-layer cognitive architecture that encapsulates key psychological mechanisms.<n>In particular, we introduce the Intimacy--Curiosity--Reciprocity--Risk (ICR2) motivational engine grounded in psychological and sociological theories.
  arXiv  Detail & Related papers  (2025-05-27T13:09:21Z)
• SOTOPIA-$Ω$: Dynamic Strategy Injection Learning and Social Instruction Following Evaluation for Social Agents [16.320531397370008]
  We propose a framework for enhancing the social capabilities of language agents.<n>We introduce the concept of Social Instruction Following (S-IF) and propose two new S-IF evaluation metrics.<n>We demonstrate that several 7B models trained on high-quality corpus not only significantly surpass the expert agent (GPT-4) in achieving social goals.
  arXiv  Detail & Related papers  (2025-02-21T15:40:37Z)
• SDPO: Segment-Level Direct Preference Optimization for Social Agents [56.970902914217156]
  Social agents powered by large language models (LLMs) can simulate human social behaviors but fall short in handling complex social dialogues.<n>We propose Segment-Level Direct Preference Optimization (SDPO), which dynamically select key segments within interactions to optimize multi-turn agent behavior.
  arXiv  Detail & Related papers  (2025-01-03T14:09:46Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.