Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization

• URL: http://arxiv.org/abs/2606.12373v1
• Date: Wed, 10 Jun 2026 17:39:41 GMT
• Title: Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization
• Abstract summary: This paper introduces RACES (textbfRecursive textbfAutomated textbfComposition for textbfEnvironment textbfScaling), a framework that conceptualizes verifiable environments as composable building blocks.<n> RACES is implemented with 300 individual environments and defines a set of composition operators that induce diverse reasoning patterns.
• Score: 93.59040719904154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement Learning (RL) with verifiable environments has emerged as a powerful approach for enhancing the reasoning capabilities of Large Language Models (LLMs). While prior research demonstrates that scaling environment quantity improves RL performance, existing manual or individual construction methods suffer from linear scaling limits, thereby hindering scalable reasoning generalization. This paper introduces RACES (\textbf{R}ecursive \textbf{A}utomated \textbf{C}omposition for \textbf{E}nvironment \textbf{S}caling), a framework that conceptualizes verifiable environments as composable building blocks that can be recursively assembled. The key insight is that when the codomain (output type) of one environment matches the domain (input type) of another, they can be automatically fused into a new verifiable environment, enabling recursive composition. RACES is implemented with 300 individual environments and defines a set of composition operators (\textsc{SEQUENTIAL}, \textsc{PARALLEL}, \textsc{SORT}, and \textsc{SELECT}) that induce diverse reasoning patterns. Extensive experiments show that RL training on these composite environments consistently enhances reasoning generalization. Specifically, RACES improves DeepSeek-R1-Distill-Qwen-14B by an average of 3.1 points (from 48.2 to 51.3) and boosts Qwen3-14B performance from 58.8 to 61.1 on six benchmarks, which are unseen during the construction of training environments. Moreover, RACES achieves performance comparable to training on 300 individual environments using only 50 base environments, demonstrating significant efficiency in environment utilization.

Related papers

• AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scale [6.489568846701505]
  We introduce AgentMercury, a framework for synthesizing executable environments from high-level business scenarios.<n>We construct 4,783 executable environments spanning 14 industries and 50 countries, and use them as training substrates for reinforcement learning.<n>We show that fine-tuning Qwen3.5-35B-A3B on construction traces increases executable authoring success from 3.3% to 83.3% on held-out business scenarios.
  arXiv  Detail & Related papers  (2026-08-21T00:15:45Z)
• SPADE: Self-Play in Adaptive Synthetic Executable Environments [110.80471334001574]
  We introduce SPADE, a self-play RL framework in which a single LLM plays two roles.<n>The Environment Designer writes complete, long-horizon training environments as executable code with an OpenAI Gym-style reset()/step() interface.<n>We find several components critical to success: grounding the Environment Designer on documents sampled from a large pretraining corpus, and giving it an accumulated environment memory.
  arXiv  Detail & Related papers  (2026-08-19T17:58:56Z)
• NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research [5.287668610154937]
  Post-training alignment determines the reasoning and human preference following capabilities of large language models.<n>This work presents NebulaExp, a fully transparent, ablation-driven post-training pipeline built on Qwen3-8B-base.
  arXiv  Detail & Related papers  (2026-06-25T07:03:25Z)
• ClawEnvKit: Automatic Environment Generation for Claw-Like Agents [85.29126619772153]
  We introduce ClawEnvKit, an autonomous generation pipeline capable of generating verified environments on demand.<n>ClawEnvKit comprises three modules: (1) a pipeline that extracts structured generation parameters from natural language input; (2) a generator that produces the task specification, tool interface, and scoring configuration; and (3) a validator that enforces feasibility, diversity, structural validity, and internal consistency.<n>We construct Auto-ClawEval, the first large-scale benchmark for claw-like agents, comprising 1,040 environments across 24 categories.
  arXiv  Detail & Related papers  (2026-04-20T17:36:49Z)
• Ambig-IaC: Multi-level Disambiguation for Interactive Cloud Infrastructure-as-Code Synthesis [8.743123758115608]
  Large Language models (LLMs) are increasingly being used to generate IaC configurations from natural language.<n>We observe that ambiguity in IaC exhibits a tractable compositional structure.<n>We propose a training-free, disagreement-driven framework that generates diverse candidate specifications.
  arXiv  Detail & Related papers  (2026-04-01T20:27:43Z)
• Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning [62.499592503950026]
  Large language model (LLM) have empowered autonomous agents to perform complex tasks that require multi-turn interactions with tools and environments.<n>We propose Agent World Model (AWM), a fully synthetic environment generation pipeline.<n>We scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets.
  arXiv  Detail & Related papers  (2026-02-10T18:55:41Z)
• RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments [111.87296453908199]
  We introduce Reinforcement Learning with Adaptive Verifiable Environments (RLVE)<n>RLVE enables each verifiable environment to dynamically adapt its problem difficulty distribution to the policy model's capabilities as training progresses.<n>We show that environment scaling, i.e., expanding the collection of training environments, consistently improves reasoning capabilities.
  arXiv  Detail & Related papers  (2025-11-10T17:18:35Z)
• Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments [70.42705564227548]
  We propose an automated environment construction pipeline for large language models (LLMs)<n>This enables the creation of high-quality training environments that provide detailed and measurable feedback without relying on external tools.<n>We also introduce a verifiable reward mechanism that evaluates both the precision of tool use and the completeness of task execution.
  arXiv  Detail & Related papers  (2025-08-12T09:45:19Z)
• GRASP: A Grid-Based Benchmark for Evaluating Commonsense Spatial Reasoning [2.9312156642007294]
  We construct a large-scale benchmark called GRASP, which consists of 16,000 grid-based environments where the agent is tasked with an energy collection problem.<n>We compare classic baseline approaches, such as random walk and greedy search methods, with advanced LLMs like GPT-3.5-Turbo, GPT-4o, and GPT-o1-mini.<n>The experimental results indicate that even advanced LLMs struggle to consistently achieve satisfactory solutions.
  arXiv  Detail & Related papers  (2024-07-02T02:27:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.