ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering

• URL: http://arxiv.org/abs/2505.23723v1
• Date: Thu, 29 May 2025 17:54:44 GMT
• Title: ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering
• Authors: Zexi Liu, Jingyi Chai, Xinyu Zhu, Shuo Tang, Rui Ye, Bo Zhang, Lei Bai, Siheng Chen,
• Abstract summary: Large language model (LLM)-based agents have significantly advanced the development of autonomous machine learning (ML) engineering.<n>We focus on the paradigm of learning-based agentic ML, where an LLM agent learns through interactive experimentation on ML tasks using online reinforcement learning (RL)<n>We propose a novel agentic ML training framework with three key components: exploration-enriched fine-tuning, step-wise RL, and an agentic ML-specific reward module.<n>Remarkably, despite being trained on merely 9 ML tasks, our 7B-sized ML-Agent outperforms the 671B-sized DeepSeek-R
• Score: 38.14868743029147
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of large language model (LLM)-based agents has significantly advanced the development of autonomous machine learning (ML) engineering. However, most existing approaches rely heavily on manual prompt engineering, failing to adapt and optimize based on diverse experimental experiences. Focusing on this, for the first time, we explore the paradigm of learning-based agentic ML, where an LLM agent learns through interactive experimentation on ML tasks using online reinforcement learning (RL). To realize this, we propose a novel agentic ML training framework with three key components: (1) exploration-enriched fine-tuning, which enables LLM agents to generate diverse actions for enhanced RL exploration; (2) step-wise RL, which enables training on a single action step, accelerating experience collection and improving training efficiency; (3) an agentic ML-specific reward module, which unifies varied ML feedback signals into consistent rewards for RL optimization. Leveraging this framework, we train ML-Agent, driven by a 7B-sized Qwen-2.5 LLM for autonomous ML. Remarkably, despite being trained on merely 9 ML tasks, our 7B-sized ML-Agent outperforms the 671B-sized DeepSeek-R1 agent. Furthermore, it achieves continuous performance improvements and demonstrates exceptional cross-task generalization capabilities.

Related papers

• Omni-Thinker: Scaling Cross-Domain Generalization in LLMs via Multi-Task RL with Hybrid Rewards [50.21528417884747]
  We introduce Omni-Thinker, a unified reinforcement learning framework that enhances large language models (LLMs) performance across diverse tasks.<n>Our approach enables consistent optimization across task types and scales RL-based training to subjective domains.<n> Experimental results across four domains reveal that curriculum learning improves performance by 5.2% over joint training and 9.1% over model merging.
  arXiv  Detail & Related papers  (2025-07-20T01:50:16Z)
• LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generation [12.098817831819078]
  Large Language Models (LLMs) have shown promise in single-robot settings, but their application in multi-robot systems remains largely unexplored.<n>This paper introduces a novel LLM-Aided MARL (LAMARL) approach, which integrates MARL with LLMs, significantly enhancing sample efficiency without requiring manual design.
  arXiv  Detail & Related papers  (2025-06-02T10:59:54Z)
• Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking [61.61356842567952]
  We propose STeP, a novel method for improving LLM-based agent training.<n>We synthesize self-reflected trajectories that include reflections and corrections of error steps.<n>Experiments demonstrate that our method improves agent performance across three representative tasks.
  arXiv  Detail & Related papers  (2025-05-26T14:11:12Z)
• MLZero: A Multi-Agent System for End-to-end Machine Learning Automation [48.716299953336346]
  We introduce MLZero, a novel multi-agent framework powered by Large Language Models (LLMs)<n>A cognitive perception module is first employed, transforming raw multimodal inputs into perceptual context.<n> MLZero demonstrates superior performance on MLE-Bench Lite, outperforming all competitors in both success rate and solution quality.
  arXiv  Detail & Related papers  (2025-05-20T05:20:53Z)
• MALT: Improving Reasoning with Multi-Agent LLM Training [66.9481561915524]
  MALT (Multi-Agent LLM Training) is a novel post-training strategy that divides the reasoning process into generation, verification, and refinement steps.<n>On MATH, GSM8K, and CSQA, MALT surpasses the same baseline LLM with a relative improvement of 15.66%, 7.42%, and 9.40% respectively.
  arXiv  Detail & Related papers  (2024-12-02T19:30:36Z)
• EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents [65.38474102119181]
  We propose EnvGen, a framework to adaptively create training environments. We train a small RL agent in a mixture of the original and LLM-generated environments. We find that a small RL agent trained with EnvGen can outperform SOTA methods, including a GPT-4 agent, and learns long-horizon tasks significantly faster.
  arXiv  Detail & Related papers  (2024-03-18T17:51:16Z)
• ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL [80.10358123795946]
  We develop a framework for building multi-turn RL algorithms for fine-tuning large language models. Our framework adopts a hierarchical RL approach and runs two RL algorithms in parallel. Empirically, we find that ArCHer significantly improves efficiency and performance on agent tasks.
  arXiv  Detail & Related papers  (2024-02-29T18:45:56Z)
• Offline Training of Language Model Agents with Functions as Learnable Weights [39.88545362699836]
  We present a novel paradigm of training Large Language Models (LLMs) agents without modifying the LLM weights. We develop Agentr that employs the LLM to update agents' functions and devise an agent training algorithm with two strategies, roll-back, and early-stop. With extensive experiments, we showcase that the agent training paradigm could significantly improve the performance of representative LLM agents.
  arXiv  Detail & Related papers  (2024-02-17T18:31:21Z)
• GEVO-ML: Optimizing Machine Learning Code with Evolutionary Computation [6.525197444717069]
  GEVO-ML is a tool for discovering optimization opportunities and tuning the performance of Machine Learning kernels. We demonstrate GEVO-ML on two different ML workloads for both model training and prediction. GEVO-ML finds significant improvements for these models, achieving 90.43% performance improvement when model accuracy is relaxed by 2%.
  arXiv  Detail & Related papers  (2023-10-16T09:24:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.