RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments

• URL: http://arxiv.org/abs/2603.16453v1
• Date: Tue, 17 Mar 2026 12:35:52 GMT
• Title: RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments
• Abstract summary: Large Language Model (LLM)-based agents have achieved notable success on short-horizon and highly structured tasks.<n>We introduce RetailBench, a high-fidelity benchmark designed to evaluate long-horizon autonomous decision-making in realistic commercial scenarios.<n>We propose the Evolving Strategy & Execution framework, which separates high-level strategic reasoning from low-level action execution.
• Score: 8.751899157366005
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Large Language Model (LLM)-based agents have achieved notable success on short-horizon and highly structured tasks. However, their ability to maintain coherent decision-making over long horizons in realistic and dynamic environments remains an open challenge. We introduce RetailBench, a high-fidelity benchmark designed to evaluate long-horizon autonomous decision-making in realistic commercial scenarios, where agents must operate under stochastic demand and evolving external conditions. We further propose the Evolving Strategy & Execution framework, which separates high-level strategic reasoning from low-level action execution. This design enables adaptive and interpretable strategy evolution over time. It is particularly important for long-horizon tasks, where non-stationary environments and error accumulation require strategies to be revised at a different temporal scale than action execution. Experiments on eight state-of-the-art LLMs across progressively challenging environments show that our framework improves operational stability and efficiency compared to other baselines. However, performance degrades substantially as task complexity increases, revealing fundamental limitations in current LLMs for long-horizon, multi-factor decision-making.

Related papers

• Agentic AI for Bilevel Long-Term Optimization of Policy-Driven Physical Layer Systems [56.931625543734306]
  Agentic-O is a nested bilevel optimization framework that can be applied to adaptive physical layer problem configuration.<n>Agentic-O exhibits strong adaptability to dynamic operator policies and effectively enhances the system's long-term performance by 57.2% compared to traditional methods.
  arXiv  Detail & Related papers  (2026-06-23T10:53:12Z)
• RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environments [8.751899157366005]
  Large language model (LLM) agents have made rapid progress on short-horizon, well-scoped tasks, yet their ability to sustain coherent decisions in dynamic long-horizon environments remains uncertain.<n>We introduce RetailBench, a data-grounded simulation benchmark for evaluating tool-using LLM agents in single-store supermarket operation.
  arXiv  Detail & Related papers  (2026-06-14T15:30:30Z)
• Beyond Scaling: Assessing Strategic Reasoning and Rapid Decision-Making Capability of LLMs in Zero-sum Environments [15.538910160052964]
  This paper introduces Strategic Tactical Agent Reasoning (STAR) Benchmark, a multi-agent evaluation framework.<n>STAR supports both turn-based and real-time settings, enabling controlled analysis of long-horizon strategic planning.<n>We show that strategic intelligence in interactive environments depends not only on reasoning depth, but also on the ability to translate plans into timely actions.
  arXiv  Detail & Related papers  (2026-03-10T08:14:13Z)
• HiMAC: Hierarchical Macro-Micro Learning for Long-Horizon LLM Agents [19.63866851076813]
  HiMAC is a hierarchical agentic RL framework that decomposes long-horizon decision-making into macro-level planning and micro-level execution.<n>Our results show that introducing structured hierarchy, rather than increasing model scale alone, is a key factor for enabling robust long-horizon agentic intelligence.
  arXiv  Detail & Related papers  (2026-03-01T08:09:03Z)
• EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies [61.267836443398124]
  We introduce EcoGym, a benchmark for continuous plan-and-execute decision making in interactive economies.<n>EcoGym is released as an open, testbed for transparent long-horizonity agent evaluation and for studying controllability-utility trade-offs in realistic economic settings.
  arXiv  Detail & Related papers  (2026-02-10T08:12:23Z)
• PILOT: Planning via Internalized Latent Optimization Trajectories for Large Language Models [51.43746425777865]
  Large Language Models (LLMs) often lack the capacity to formulate global strategies, leading to error propagation in long-horizon tasks.<n>We propose PILOT, a framework designed to internalize the strategic oversight of large models into intrinsic Latent Guidance.
  arXiv  Detail & Related papers  (2026-01-07T12:38:56Z)
• Learning Affordances at Inference-Time for Vision-Language-Action Models [50.93181349331096]
  In robotics, Vision-Language-Action models (VLAs) offer a promising path towards solving complex control tasks.<n>We introduce Learning from Inference-Time Execution (LITEN), which connects a VLA low-level policy to a high-level VLM that conditions on past experiences.<n>Our approach iterates between a reasoning phase that generates and executes plans for the low-level VLA, and an assessment phase that reflects on the resulting execution.
  arXiv  Detail & Related papers  (2025-10-22T16:43:29Z)
• HeroBench: A Benchmark for Long-Horizon Planning and Structured Reasoning in Virtual Worlds [0.0]
  Large language models (LLMs) have shown remarkable capabilities in isolated step-by-step reasoning tasks such as mathematics and programming.<n>But their proficiency in long-horizon planning, where solutions require extended, structured sequences of interdependent actions, remains underexplored.<n>We introduce HeroBench, a novel benchmark designed specifically to evaluate long-horizon planning and structured reasoning within complex RPG-inspired virtual worlds.
  arXiv  Detail & Related papers  (2025-08-18T09:59:02Z)
• Structured Preference Optimization for Vision-Language Long-Horizon Task Planning [72.60833832298343]
  Existing methods for vision-language task planning excel in short-horizon tasks but often fall short in complex, long-horizon planning within dynamic environments.<n>These challenges arise from the difficulty of effectively training models to produce high-quality reasoning processes for long-horizon tasks.<n>We propose Structured Preference Optimization (SPO), which aims to enhance reasoning and action selection in long-horizon task planning.
  arXiv  Detail & Related papers  (2025-02-28T05:47:34Z)
• EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning [69.55982246413046]
  We propose explicit policy optimization (EPO) for strategic reasoning.<n>We train the strategic reasoning model via multi-turn reinforcement learning (RL),utilizing process rewards and iterative self-play.<n>Our findings reveal various collaborative reasoning mechanisms emergent in EPO and its effectiveness in generating novel strategies.
  arXiv  Detail & Related papers  (2025-02-18T03:15:55Z)
• STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making [43.734386326024016]
  Large Language Models (LLMs) have revolutionized natural language processing, showing remarkable linguistic proficiency and reasoning capabilities. This paper presents a novel framework equipped with memory and specialized tools to enhance their strategic decision-making capabilities.
  arXiv  Detail & Related papers  (2024-05-25T23:25:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.