MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games

• URL: http://arxiv.org/abs/2603.09022v1
• Date: Mon, 09 Mar 2026 23:36:32 GMT
• Title: MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games
• Abstract summary: Multi-agent game evaluations often exhibit substantial run-to-run variance.<n>Small deviations compound across turns and are amplified by multi-agent coupling.<n>We address both instability and underperformance with MEMO (Memory-augmented MOdel context optimization), a self-play framework.
• Score: 79.72300527041534
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Multi-turn, multi-agent LLM game evaluations often exhibit substantial run-to-run variance. In long-horizon interactions, small early deviations compound across turns and are amplified by multi-agent coupling. This biases win rate estimates and makes rankings unreliable across repeated tournaments. Prompt choice worsens this further by producing different effective policies. We address both instability and underperformance with MEMO (Memory-augmented MOdel context optimization), a self-play framework that optimizes inference-time context by coupling retention and exploration. Retention maintains a persistent memory bank that stores structured insights from self-play trajectories and injects them as priors during later play. Exploration runs tournament-style prompt evolution with uncertainty-aware selection via TrueSkill, and uses prioritized replay to revisit rare and decisive states. Across five text-based games, MEMO raises mean win rate from 25.1% to 49.5% for GPT-4o-mini and from 20.9% to 44.3% for Qwen-2.5-7B-Instruct, using $2,000$ self-play games per task. Run-to-run variance also drops, giving more stable rankings across prompt variations. These results suggest that multi-agent LLM game performance and robustness have substantial room for improvement through context optimization. MEMO achieves the largest gains in negotiation and imperfect-information games, while RL remains more effective in perfect-information settings.

Related papers

• Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory [19.236924182307085]
  Existing kernel-optimization agents seldom preserve a decision, its observed execution feedback, and the later decisions that use that evidence.<n>We present KOPE, an experience-driven framework for hardware kernel optimization.
  arXiv  Detail & Related papers  (2026-08-26T09:27:39Z)
• Towards Improving Sequential Decision-Making in LLM Agents via Experience Memory [3.4935179780034242]
  Large language models have improved substantially on single-shot reasoning tasks, but their performance in sequential decision-making is less well understood.<n>We study this on fully-observable two-player zero-sum games, which provide ground-truth evaluation.<n>We show that post-game reflection and rule extraction yield measurable improvements on tic-tac-toe without modifying the model weights.
  arXiv  Detail & Related papers  (2026-08-04T10:12:12Z)
• MAGE: Understanding Stability-Performance Trade-offs in Multi-component Prompt Optimization [3.7323766833175718]
  MAGE (Memory-Augmented Goal Prompt Evolution) is a controlled analysis framework for studying component interaction in prompt optimization.<n>Our experiments uncover a previously unreported phenomenon, the Prompt Optimization Coupling Effect (POCE), when multiple optimization signals operate within a closed reflective loop.
  arXiv  Detail & Related papers  (2026-07-11T10:05:58Z)
• From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory [27.810603317768997]
  Large language model (LLM) agents are increasingly deployed in long-running settings where improving through experience at test time is important.<n>We propose MemoPilot, a plug-in memory copilot that explicitly trains the memory update process to improve a frozen LLM's performance across sequential interactions.<n>We evaluate MemoPilot on two testbeds: multi-round Rock-Paper-Scissors (RPS) and Limit Texas Hold'em (LHE)
  arXiv  Detail & Related papers  (2026-06-07T14:53:19Z)
• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs [54.81359054218573]
  We introduce Mindgames, a multi-game arena and evaluation platform for large language models (LLMs)<n>Mindgames provides a unified interaction interface, TrueSkill-based rating, and full trajectory logging across four game environments.<n>We analyze 29,571 multi-agent games with turn-level observations, actions, and rewards, together with MG-Ref, a deterministic offline tournament protocol.
  arXiv  Detail & Related papers  (2026-05-28T07:33:47Z)
• MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning [73.27233666920618]
  We propose MemSearcher, an agent workflow that iteratively maintains a compact memory and combines the current turn with it.<n>At each turn, MemSearcher fuses the user's question with the memory to generate reasoning traces, perform search actions, and update memory to retain only information essential for solving the task.<n>We introduce multi-context GRPO, an end-to-end RL framework that jointly optimize reasoning, search strategies, and memory management of MemSearcher Agents.
  arXiv  Detail & Related papers  (2025-11-04T18:27:39Z)
• Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games [60.213483076150844]
  We propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL.<n>We conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning.<n>We achieve a 54.76% win rate against GPT-4 in six adversarial games.
  arXiv  Detail & Related papers  (2025-10-19T09:03:28Z)
• Reinforcing Video Reasoning with Focused Thinking [65.85683941058916]
  We propose TW-GRPO, a novel framework that enhances visual reasoning with focused thinking and dense reward granularity.<n>Specifically, we employ a token weighting mechanism that prioritizes tokens with high informational density.<n>We also reformulate RL training by shifting from single-choice to multi-choice QA tasks.
  arXiv  Detail & Related papers  (2025-05-30T15:42:19Z)
• Auto-Encoding Bayesian Inverse Games [36.06617326128679]
  We consider the inverse game problem, in which some properties of the game are unknown a priori. Existing maximum likelihood estimation approaches to solve inverse games provide only point estimates of unknown parameters. We take a Bayesian perspective and construct posterior distributions of game parameters. This structured VAE can be trained from an unlabeled dataset of observed interactions.
  arXiv  Detail & Related papers  (2024-02-14T02:17:37Z)
• MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimization [12.473095790918347]
  We propose name, which formulates optimal prioritized experience replay for multi-agent problems. By minimizing the resulting policy regret, we can narrow the gap between the current policy and a nominal optimal policy.
  arXiv  Detail & Related papers  (2023-02-21T03:11:21Z)
• Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions [145.54544979467872]
  We propose and analyze new fictitious play policy optimization algorithms for zero-sum Markov games with structured but unknown transitions. We prove tight $widetildemathcalO(sqrtK)$ regret bounds after $K$ episodes in a two-agent competitive game scenario. Our algorithms feature a combination of Upper Confidence Bound (UCB)-type optimism and fictitious play under the scope of simultaneous policy optimization.
  arXiv  Detail & Related papers  (2022-07-25T18:29:16Z)
• Off-Beat Multi-Agent Reinforcement Learning [62.833358249873704]
  We investigate model-free multi-agent reinforcement learning (MARL) in environments where off-beat actions are prevalent. We propose a novel episodic memory, LeGEM, for model-free MARL algorithms. We evaluate LeGEM on various multi-agent scenarios with off-beat actions, including Stag-Hunter Game, Quarry Game, Afforestation Game, and StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2022-05-27T02:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.