L0: Reinforcement Learning to Become General Agents

• URL: http://arxiv.org/abs/2506.23667v1
• Date: Mon, 30 Jun 2025 09:44:32 GMT
• Title: L0: Reinforcement Learning to Become General Agents
• Abstract summary: We introduce L-Zero (L0), a scalable, end-to-end training pipeline for general-purpose agents.<n> NB-Agent, the agent scaffold within L0, operates in a "code-as-action" fashion via a Read-Eval-Print-Loop (REPL)<n>Experiments demonstrate that a base model can develop robust problem-solving skills using solely Reinforcement Learning with Verifiable Rewards (RLVR)
• Score: 9.210885263681119
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Training large language models (LLMs) to act as autonomous agents for multi-turn, long-horizon tasks remains significant challenges in scalability and training efficiency. To address this, we introduce L-Zero (L0), a scalable, end-to-end training pipeline for general-purpose agents. Featuring a low-cost, extensible, and sandboxed concurrent agent worker pool, L0 lowers the barrier for applying reinforcement learning in complex environments. We also introduce NB-Agent, the agent scaffold within L0, which operates in a "code-as-action" fashion via a Read-Eval-Print-Loop (REPL). We evaluate L0 on factuality question-answering benchmarks. Our experiments demonstrate that a base model can develop robust problem-solving skills using solely Reinforcement Learning with Verifiable Rewards (RLVR). On the Qwen2.5-7B-Instruct model, our method boosts accuracy on SimpleQA from 30 % to 80 % and on HotpotQA from 22 % to 41 %. We have open-sourced the entire L0 system, including our L0 series models, the NB-Agent, a complete training pipeline, and the corresponding training recipes on (https://github.com/cmriat/l0).

Related papers

• LLMs Are Already Good Tutors: Training-Free Prompt Optimization for Pedagogical Math Tutoring [1.4976621159818981]
  We investigate whether training-free prompt optimization-evolving only the system prompt via API calls can serve as a practical alternative.<n>We adapt 7 published methods and propose 5 education-specialized methods, evaluating these 12 methods under 5 conditions on 2 OOD benchmark suites.<n> Behavioral analysis with an 82-code educational codebook reveals that training-free methods rely on teaching-knowledge patterns at 2-3x the rate of RL-trained models.
  arXiv  Detail & Related papers  (2026-05-26T14:35:57Z)
• ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation [55.947962672433675]
  ChipMATE is the first self-trained multi-agent framework for RTL generation.<n>Inspired by industrial practice, ChipMATE pairs a Verilog agent with a Python reference-model agent that mutually verify each other's outputs.<n>ChipMATE achieves 75.0% and 80.1% pass@1 on VerilogEval V2 with 4B and 9B base models.
  arXiv  Detail & Related papers  (2026-05-13T01:04:21Z)
• RefineRL: Advancing Competitive Programming with Self-Refinement Reinforcement Learning [63.432969627395686]
  RefineRL is a novel approach designed to unleash the self-refinement capabilities of large language models for competitive programming problems.<n>Skeptical-Agent is an iterative self-refinement agent equipped with local execution tools to validate generated solutions against public test cases of CP problems.<n>A reinforcement learning solution incentivizes LLMs to self-refine with only standard RLVR data.
  arXiv  Detail & Related papers  (2026-04-01T11:54:57Z)
• Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data [49.315842374696295]
  Large language models (LLMs) are becoming the foundation for autonomous agents that can use tools to solve complex tasks.<n>We propose Tool-R0 framework for training general purpose tool-calling agents from scratch with self-play RL.<n>Our work further provides empirical insights into self-play LLM agents by analyzing co-evolution, curriculum dynamics, and scaling behavior.
  arXiv  Detail & Related papers  (2026-02-24T19:41:18Z)
• Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning [84.70211451226835]
  Large Language Model (LLM) Agents are constrained by a dependency on human-curated data.<n>We introduce Agent0, a fully autonomous framework that evolves high-performing agents without external data.<n>Agent0 substantially boosts reasoning capabilities, improving the Qwen3-8B-Base model by 18% on mathematical reasoning and 24% on general reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-11-20T05:01:57Z)
• Internalizing World Models via Self-Play Finetuning for Agentic RL [65.96875390986655]
  Large Language Models (LLMs) as agents often struggle in out-of-distribution (OOD) scenarios.<n>We show how to encode this world model by decomposing it into two components: state representation and transition modeling.<n>We introduce SPA, a simple reinforcement learning framework that cold-starts the policy via a Self-Play supervised finetuning stage to learn the world model.
  arXiv  Detail & Related papers  (2025-10-16T18:03:39Z)
• R-Zero: Self-Evolving Reasoning LLM from Zero Data [56.74402018426378]
  Self-evolving Large Language Models (LLMs) offer a scalable path toward super-intelligence by autonomously generating, refining, and learning from their own experiences.<n>Existing methods for training such models still rely heavily on vast human-curated tasks and labels.<n>We introduce R-Zero, a fully autonomous framework that generates its own training data from scratch.
  arXiv  Detail & Related papers  (2025-08-07T03:38:16Z)
• How to Train Your LLM Web Agent: A Statistical Diagnosis [102.04125085041473]
  We present the first statistically grounded study on compute allocation for LLM web-agent post-training.<n>Our approach uses a two-stage pipeline, training a Llama 3.1 8B student to imitate a Llama 3.3 70B teacher via supervised fine-tuning (SFT) and on-policy reinforcement learning.<n>Our results show that combining SFT with on-policy RL consistently outperforms either approach alone on both WorkArena and MiniWob++.
  arXiv  Detail & Related papers  (2025-07-05T17:12:33Z)
• Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models [15.218318229687242]
  Extreme activation outliers in Large Language Models critically degrade quantization performance.<n>We introduce Outlier-Safe Pre-Training (OSP), a practical guideline that proactively prevents formation.<n>Our work demonstrates that outliers are not inherent to LLMs but are consequences of training strategies.
  arXiv  Detail & Related papers  (2025-06-24T15:03:57Z)
• Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training [69.13064064991552]
  Hephaestus-Forge is a large-scale pre-training corpus designed to enhance the capabilities of LLM agents in API function calling, intrinsic reasoning and planning.<n>Hephaestus-Forge comprises 103B agent-specific data encompassing 76,537 APIs, including both tool documentation to introduce knowledge of API functions and function calling trajectories.<n>By continual pre-training on Hephaestus-Forge, Hephaestus outperforms small- to medium-scale open-source LLMs and rivals commercial LLMs on three agent benchmarks.
  arXiv  Detail & Related papers  (2025-02-10T15:54:34Z)
• Reinforcement Learning for Long-Horizon Interactive LLM Agents [56.9860859585028]
  Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests.<n>We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments.<n>We derive LOOP, a data- and memory-efficient variant of proximal policy optimization.
  arXiv  Detail & Related papers  (2025-02-03T18:35:42Z)
• Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Framework [1.5802986215292307]
  Language Model Guided reward Tuning (LMGT) is a novel, sample-efficient framework for Reinforcement Learning.<n>We show that LMGT adeptly balances exploration and exploitation, thereby guiding the agent's exploratory behavior.<n>Our results suggest that LMGT can substantially reduce the computational resources required during the RL training phase.
  arXiv  Detail & Related papers  (2024-09-07T07:40:43Z)
• FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation [32.01836613286288]
  This work presents a Fully BInarized Large Language Model (FBI-LLM) It demonstrates for the first time how to train a large-scale binary language model from scratch.
  arXiv  Detail & Related papers  (2024-07-09T17:59:48Z)
• DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning [56.887047551101574]
  We present DS-Agent, a novel framework that harnesses large language models (LLMs) agent and case-based reasoning (CBR) In the development stage, DS-Agent follows the CBR framework to structure an automatic iteration pipeline, which can flexibly capitalize on the expert knowledge from Kaggle. In the deployment stage, DS-Agent implements a low-resource deployment stage with a simplified CBR paradigm, significantly reducing the demand on foundational capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-02-27T12:26:07Z)
• MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT [87.4910758026772]
  "Bigger the better" has been the predominant trend in recent Large Language Models (LLMs) development. This paper explores the "less is more" paradigm by addressing the challenge of designing accurate yet efficient Small Language Models (SLMs) for resource constrained devices.
  arXiv  Detail & Related papers  (2024-02-26T18:59:03Z)
• Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels [112.63440666617494]
  Reinforcement learning algorithms can succeed but require large amounts of interactions between the agent and the environment. We propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent. We show robust performance on the Real-Word RL benchmark, hinting at resiliency to environment perturbations during adaptation.
  arXiv  Detail & Related papers  (2022-09-24T14:22:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.