State2State: Environment-Derived Mid-Training for LLM Agents

• URL: http://arxiv.org/abs/2608.04934v1
• Date: Wed, 05 Aug 2026 15:02:41 GMT
• Title: State2State: Environment-Derived Mid-Training for LLM Agents
• Abstract summary: We study an environment learning paradigm in which agents acquire interaction and manipulation capabilities solely through environment interaction.<n>We propose State2State, an environment-derived mid-training method that converts explored environment states into training objectives.<n> Experiments on ALFWorld and ScienceWorld show that State2State improves agent performance as a standalone environment-learning stage in most settings.
• Score: 67.08889234341996
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training LLM agents commonly relies on supervised fine-tuning from expert trajectories or online reinforcement learning over human-specified tasks with handcrafted verifiers. Though effective, both remain bottlenecked by externally specified tasks and supervision signals, limiting the scalability and diversity of agent training. We study an environment learning paradigm in which agents acquire interaction and manipulation capabilities solely through environment interaction, without externally specified tasks. We propose State2State, an environment-derived mid-training method that converts explored environment states into training objectives, challenging agents to reach a specified target state. By deriving tasks from environment exploration and verifying success through rule-based state matching, State2State provides scalable and verifiable training objectives without expert supervision or manual task design. Experiments on ALFWorld and ScienceWorld show that State2State improves agent performance as a standalone environment-learning stage in most settings. As initialization for downstream RL, it further improves final performance and learning efficiency, with promising evidence of cross-environment generalization.

Related papers

• Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence [122.8626256816085]
  textbfAgent-World is a self-evolving training arena for advancing general agent intelligence through scalable environments.<n>Agent-World-8B and 14B consistently outperforms strong proprietary models and environment scaling baselines.
  arXiv  Detail & Related papers  (2026-04-20T14:01:10Z)
• Can RL Improve Generalization of LLM Agents? An Empirical Study [68.19349692042341]
  Reinforcement fine-tuning (RFT) has shown promise for training LLM agents to perform multi-turn decision-making based on environment feedback.<n>In real-world deployment, agents may operate in unseen environments with different background knowledge.<n>We show that RFT generalizes well across task difficulty within an environment, but exhibits weaker transfer to unseen environments.
  arXiv  Detail & Related papers  (2026-03-12T14:54:59Z)
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation [57.65688895630163]
  We introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data.<n>Our method effectively enables both intra-environment and cross-environment continual learning, yielding 4-22% performance gains without forgetting existing environments.
  arXiv  Detail & Related papers  (2026-02-10T23:06:02Z)
• What Do LLM Agents Know About Their World? Task2Quiz: A Paradigm for Studying Environment Understanding [50.35012849818872]
  Large language model (LLM) agents have demonstrated remarkable capabilities in complex decision-making and tool-use tasks.<n>We propose Task-to-Quiz (T2Q), a deterministic and automated evaluation paradigm designed to decouple task execution from world-state understanding.<n>Our experiments reveal that task success is often a poor proxy for environment understanding, and that current memory machanism can not effectively help agents acquire a grounded model of the environment.
  arXiv  Detail & Related papers  (2026-01-14T14:09:11Z)
• The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios [34.25281365374991]
  We introduce method, a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting.<n>Unlike traditional benchmarks, method evaluates agents along three dimensions: (1) context-aware scheduling for streaming tasks with varying priorities; (2) prudent information acquisition to reduce hallucination via active exploration; and (3) continuous evolution by distilling generalized strategies from rule-based, dynamically generated tasks.<n>Our work establishes a framework for assessing agent reliability, shifting evaluation from static tests to realistic, production-oriented scenarios.
  arXiv  Detail & Related papers  (2026-01-13T03:09:18Z)
• Agent Learning via Early Experience [93.83579011718858]
  A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks.<n>Most current agents rely on supervised fine-tuning on expert data, which is challenging to scale and generalizes poorly.<n>We study two strategies of using such data: (1) Implicit world modeling, which uses collected states to ground the policy in environment dynamics; and (2) Self-reflection, where the agent learns from its suboptimal actions to improve reasoning and decision-making.
  arXiv  Detail & Related papers  (2025-10-09T17:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.