LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning

• URL: http://arxiv.org/abs/2608.12626v1
• Date: Wed, 12 Aug 2026 22:17:24 GMT
• Title: LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning
• Abstract summary: EpicStar is a framework that enables agents to learn memory as policy to tackle long-horizon reasoning.<n>During inference, a dynamic gating mechanism determines whether to execute a retrieved action directly or to perform new reasoning.<n>It significantly outperforms baseline methods, achieving higher win rates while consuming an order of magnitude fewer tokens.
• Score: 4.794581688428893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Strategic reasoning in Large Language Models (LLMs) within long-horizon environments is often limited by inconsistent subgoals. In these settings, finite attention resources prevent the model from maintaining strategic coherence over thousands of steps. This limitation leads to strategic drift, where localized decisions fail to sustain a coherent trajectory across reasoning. To address this, we introduce EpicStar, a framework that enables agents to learn memory as policy to tackle long-horizon reasoning. Specifically, the agent maintains a bank of successful past episodes as a heuristic alongside a working memory to track short-term environmental changes. During inference, a dynamic gating mechanism determines whether to execute a retrieved action directly or to perform new reasoning through a contextual fusion of the retrieved episodes and current working memory. Utilizing StarCraft II as the testbed, we evaluated EpicStar against diverse opponent styles. It significantly outperforms baseline methods, achieving higher win rates while consuming an order of magnitude fewer tokens, and it maintains this advantage consistently across difficulty levels and opponent strategies. Our findings provide compelling evidence that structured cross-episode memory is essential for enabling LLM agents to perform robust, long-term strategic execution in dynamic, autonomous settings.

Related papers

• Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks [56.45762972211923]
  We present a proactive defense framework for securing LLMs against evolving multi-turn adversarial attacks.<n>It employs a cooperative multi-agent architecture in which specialized agents execute complementary defense strategies.<n>We present the EMRA dataset designed to simulate evolving strategies across multi-turn attacks.
  arXiv  Detail & Related papers  (2026-07-31T14:04:49Z)
• SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent [51.274445160155864]
  Long-horizon agentic reasoning requires large language models to act over long interaction histories containing thoughts, tool calls, observations, and partial conclusions.<n>Existing approaches address this difficulty by truncating the interaction history, compressing it into shorter surrogates, or retrieving selected parts of it for reuse.<n>We propose State-Adaptive Memory, a standalone framework that consolidates ongoing interaction into compact memory cues while preserving raw trajectory pages for intent-driven recall.
  arXiv  Detail & Related papers  (2026-05-23T08:37:16Z)
• Beyond Scaling: Assessing Strategic Reasoning and Rapid Decision-Making Capability of LLMs in Zero-sum Environments [15.538910160052964]
  This paper introduces Strategic Tactical Agent Reasoning (STAR) Benchmark, a multi-agent evaluation framework.<n>STAR supports both turn-based and real-time settings, enabling controlled analysis of long-horizon strategic planning.<n>We show that strategic intelligence in interactive environments depends not only on reasoning depth, but also on the ability to translate plans into timely actions.
  arXiv  Detail & Related papers  (2026-03-10T08:14:13Z)
• Controllable LLM Reasoning via Sparse Autoencoder-Based Steering [66.36947132041657]
  Large Reasoning Models (LRMs) exhibit human-like cognitive reasoning strategies.<n>Currently, reasoning strategies are autonomously selected by LRMs themselves.<n>Existing methods struggle to control fine-grained reasoning strategies due to conceptual entanglement in LRMs' hidden states.
  arXiv  Detail & Related papers  (2026-01-07T05:26:26Z)
• Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents [57.38404718635204]
  Large language model (LLM) agents face fundamental limitations in long-horizon reasoning due to finite context windows.<n>Existing methods typically handle long-term memory (LTM) and short-term memory (STM) as separate components.<n>We propose Agentic Memory (AgeMem), a unified framework that integrates LTM and STM management directly into the agent's policy.
  arXiv  Detail & Related papers  (2026-01-05T08:24:16Z)
• Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks [23.201035830828726]
  Large Language Models face challenges in long-horizon agentic tasks.<n>Existing working memory methods rely on external mechanisms that are decoupled from the agent's core policy.<n>We propose a novel framework, Memory-as-Action, where an agent actively manages its working memory by executing explicit editing operations as part of a unified policy.
  arXiv  Detail & Related papers  (2025-10-14T15:29:57Z)
• Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning [16.35236123729838]
  We propose a hierarchical multi-agent framework that employs specialized imitation learning agents under a meta-controller called Strategic Planner (SP)<n>By expert demonstrations, each specialized agent learns a distinctive strategy, such as aerial support or defensive maneuvers, and produces coherent, structured multistep action sequences.<n>The SP then orchestrates these proposals into a single, environmentally adaptive plan that ensures local decisions align with long-term strategies.
  arXiv  Detail & Related papers  (2025-08-08T05:57:12Z)
• Agents of Change: Self-Evolving LLM Agents for Strategic Planning [28.172006841163938]
  HexMachina is a continual learning multi-agent system that separates environment discovery from strategy improvement.<n>In controlled Catanatron experiments, HexMachina learns from scratch and evolves players that outperform the strongest human-crafted baseline.
  arXiv  Detail & Related papers  (2025-06-05T05:45:24Z)
• EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning [69.55982246413046]
  We propose explicit policy optimization (EPO) for strategic reasoning.<n>We train the strategic reasoning model via multi-turn reinforcement learning (RL),utilizing process rewards and iterative self-play.<n>Our findings reveal various collaborative reasoning mechanisms emergent in EPO and its effectiveness in generating novel strategies.
  arXiv  Detail & Related papers  (2025-02-18T03:15:55Z)
• Paths to Equilibrium in Games [6.812247730094933]
  We study sequences of strategies satisfying a pairwise constraint inspired by policy updating in reinforcement learning. Our analysis reveals a counterintuitive insight that reward deteriorating strategic updates are key to driving play to equilibrium along a satisficing path.
  arXiv  Detail & Related papers  (2024-03-26T19:58:39Z)
• K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning [76.3114831562989]
  It requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments. We propose a novel framework: "K-Level Reasoning with Large Language Models (K-R)"
  arXiv  Detail & Related papers  (2024-02-02T16:07:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.