Learning Simple Test-Time Environments for LLM Web Agents

• URL: http://arxiv.org/abs/2608.29305v1
• Date: Sat, 29 Aug 2026 14:45:09 GMT
• Title: Learning Simple Test-Time Environments for LLM Web Agents
• Abstract summary: We introduce trial steps for agents to decompose a complex environment observation into sub-modules.<n>We implement a label-free learning method, Test-Time Environment Decomposition (TTED), to adapt agent behaviors with experience during inference.
• Score: 25.206333684249987
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents have demonstrated remarkable proficiency in manually constructed environments, yet their performance frequently collapses when transitioned to complex real-world settings. Existing research largely attribute this degradation to the compositional generalization gaps in LLMs on combinations of multiple simple, well-structured environments. In this work, we propose that LLM web agents can learn simple environment observations at test time. Specifically, we introduce trial steps for agents to decompose a complex environment observation into sub-modules, and implement a label-free learning method, Test-Time Environment Decomposition (TTED), to adapt agent behaviors with experience during inference. Our empirical evaluations demonstrate the framework's efficacy across both synthetic and realistic benchmarks, showing (1) experience gains acquired within simpler sub-environments can be effectively composed to improve performance in the full one, and (2) test-time training on sub-environments can significantly enhance the compositional generalization of agents in real-world web automation tasks. We also provide key insights in the design of the label-free learning algorithm. As more complex environments are accessed by LLM agents, we believe learning environment decomposition skills at test time will be critical for robust real-world deployment.

Related papers

• Learning POMDP World Models from Observations with Language-Model Priors [52.11151769796862]
  Partially-observable Markov decision processes (POMDPs) provide a flexible modeling class for internal world models.<n>We ask whether language-model priors can reduce costly interaction by leveraging prior knowledge.<n>We show that emphPinductor matches the performance and sample efficiency of LLM-based POMDP learning methods.
  arXiv  Detail & Related papers  (2026-05-13T16:18:15Z)
• LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks [58.3639630490749]
  We introduce LiveClawBench, a benchmark to evaluate LLM agents on real-world assistant tasks.<n>Based on an analysis of various realClaw usage cases, we derive a Triple-Axis Complexity Framework.<n>We construct a pilot benchmark with explicit complexity-factor annotations, covering real-world assistant tasks.
  arXiv  Detail & Related papers  (2026-03-20T16:08:21Z)
• Can RL Improve Generalization of LLM Agents? An Empirical Study [68.19349692042341]
  Reinforcement fine-tuning (RFT) has shown promise for training LLM agents to perform multi-turn decision-making based on environment feedback.<n>In real-world deployment, agents may operate in unseen environments with different background knowledge.<n>We show that RFT generalizes well across task difficulty within an environment, but exhibits weaker transfer to unseen environments.
  arXiv  Detail & Related papers  (2026-03-12T14:54:59Z)
• AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning [71.4322853508083]
  Conducting reinforcement learning in simulated environments offers a cost-effective and highly scalable way to enhance language-based agents.<n>Previous work has been limited to semi-automated environment synthesis or tasks lacking sufficient difficulty, offering little breadth or depth.<n>We propose a unified pipeline for automated and scalable synthesis of simulated environments associated with high-difficulty but easily verifiable tasks.
  arXiv  Detail & Related papers  (2025-12-28T09:43:11Z)
• Scaling Environments for LLM Agents in the Era of Learning from Interaction: A Survey [30.673419015614233]
  A growing consensus is that agents should interact directly with environments and learn from experience through reinforcement learning.<n>We formalize this iterative process as the Generation-Execution-Feedback (GEF) loop, where environments generate tasks to challenge agents, return observations in response to agents' actions during task execution, and provide evaluative feedback on rollouts for subsequent learning.<n>Under this paradigm, environments function as indispensable producers of experiential data, highlighting the need to scale them toward greater complexity, realism, and interactivity.
  arXiv  Detail & Related papers  (2025-11-12T12:56:25Z)
• Grounded Test-Time Adaptation for LLM Agents [75.62784644919803]
  Large language model (LLM)-based agents struggle to generalize to novel and complex environments.<n>We propose two strategies for adapting LLM agents by leveraging environment-specific information available during deployment.
  arXiv  Detail & Related papers  (2025-11-06T22:24:35Z)
• EMAC+: Embodied Multimodal Agent for Collaborative Planning with VLM+LLM [12.54425217318534]
  We introduce EMAC+, an Embodied Multimodal Agent that collaboratively integrates LLM and VLM.<n>Unlike existing methods, EMAC+ dynamically refines high-level textual plans using real-time feedback from a VLM executing low-level visual control tasks.<n>EMAC+ achieves superior task performance, against noisy observations, and efficient learning.
  arXiv  Detail & Related papers  (2025-05-26T12:34:16Z)
• MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering [57.156093929365255]
  Gym-style framework for systematically reinforcement learning, evaluating, and improving autonomous large language model (LLM) agents.<n>MLE-Dojo covers diverse, open-ended MLE tasks carefully curated to reflect realistic engineering scenarios.<n>Its fully executable environment supports comprehensive agent training via both supervised fine-tuning and reinforcement learning.
  arXiv  Detail & Related papers  (2025-05-12T17:35:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.