From AI Assistant to AI Scientist: Autonomous Discovery of LLM-RL Algorithms with LLM Agents

• URL: http://arxiv.org/abs/2603.23951v1
• Date: Wed, 25 Mar 2026 05:19:23 GMT
• Title: From AI Assistant to AI Scientist: Autonomous Discovery of LLM-RL Algorithms with LLM Agents
• Abstract summary: POISE is a framework for automated discovery of policy optimization algorithms for language models.<n>PoISE maintains a structured, genealogically linked archive linking proposals, executable implementations, standardized evaluations, and natural-language reflections.
• Score: 51.12380889298618
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Discovering improved policy optimization algorithms for language models remains a costly manual process requiring repeated mechanism-level modification and validation. Unlike simple combinatorial code search, this problem requires searching over algorithmic mechanisms tightly coupled with training dynamics while reusing empirical evidence across iterations. We propose POISE, a closed-loop framework for automated discovery of policy optimization algorithms for language models. POISE maintains a structured, genealogically linked archive linking proposals, executable implementations, standardized evaluations, and natural-language reflections to support evidence-driven iteration. In mathematical reasoning experiments starting from GRPO, POISE evaluates 64 candidate algorithms and discovers improved mechanisms, including analytic-variance scaling and validity masking. The best variant improves weighted Overall from 47.8 to 52.5 (+4.6) and increases AIME25 pass@32 from 26.7% to 43.3%, demonstrating the feasibility of automated policy optimization discovery while supporting interpretable design principles.

Related papers

• MAS-Algorithm: A Workflow for Solving Algorithmic Programming Problems with a Multi-Agent System [15.213043425523287]
  MAS-Algorithm is a systematic multi-agent workflow for algorithmic problem solving inspired by competitive programmers and algorithm engineers.<n>Our framework decomposes the end-to-end solving process into modular stages, enabling structured reasoning, tool integration, and flexible coordination among agents.
  arXiv  Detail & Related papers  (2026-05-07T09:57:53Z)
• ContextEvolve: Multi-Agent Context Compression for Systems Code Optimization [8.046059974853858]
  We introduce ContextEvolve, a multi-agent framework that achieves RL-level search efficiency under strict parameter-blind constraints.<n>On the ADRS benchmark, ContextEvolve outperforms state-of-the-art baselines by 33.3% while reducing token consumption by 29.0%.
  arXiv  Detail & Related papers  (2026-02-01T16:50:07Z)
• QiMeng-NeuComBack: Self-Evolving Translation from IR to Assembly Code [52.66657751895655]
  Large Language Models (LLMs) offer a compelling new paradigm: Neural Compilation.<n>This paper introduces NeuComBack, a novel benchmark dataset specifically designed for IR-to-assembly compilation.<n>We propose a self-evolving prompt optimization method that enables LLMs to evolve their internal prompt strategies.
  arXiv  Detail & Related papers  (2025-11-03T03:20:26Z)
• Automated Repair of C Programs Using Large Language Models [0.0]
  This study explores the potential of Large Language Models (LLMs) in automating the repair of C programs.<n>We present a framework that integrates spectrum-based fault localization (SBFL), runtime feedback, and Chain-of-Thought-structured prompting into an autonomous repair loop.<n>Our approach achieves 44.93% repair accuracy, representing a 3.61% absolute improvement over strong state-of-the-art APR baselines.
  arXiv  Detail & Related papers  (2025-09-02T04:34:11Z)
• Evolution of Optimization Algorithms for Global Placement via Large Language Models [18.373855320220887]
  This paper presents an automated framework to evolve optimization algorithms for global placement.<n>We first generate diverse candidate algorithms using large language models (LLM) through carefully crafted prompts.<n>The discovered optimization algorithms exhibit substantial performance improvements across many benchmarks.
  arXiv  Detail & Related papers  (2025-04-18T09:57:14Z)
• RL-finetuning LLMs from on- and off-policy data with a single algorithm [53.70731390624718]
  We introduce a novel reinforcement learning algorithm (AGRO) for fine-tuning large-language models.<n>AGRO leverages the concept of generation consistency, which states that the optimal policy satisfies the notion of consistency across any possible generation of the model.<n>We derive algorithms that find optimal solutions via the sample-based policy gradient and provide theoretical guarantees on their convergence.
  arXiv  Detail & Related papers  (2025-03-25T12:52:38Z)
• DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal [55.13854171147104]
  Large Language Models (LLMs) have revolutionized various domains, including natural language processing, data analysis, and software development.<n>We present Dynamic Action Re-Sampling (DARS), a novel inference time compute scaling approach for coding agents.<n>We evaluate our approach on SWE-Bench Lite benchmark, demonstrating that this scaling strategy achieves a pass@k score of 55% with Claude 3.5 Sonnet V2.
  arXiv  Detail & Related papers  (2025-03-18T14:02:59Z)
• LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics [0.023020018305241332]
  This paper introduces a novel Large Language Model Evolutionary Algorithm (LLaMEA) framework.<n>Given a set of criteria and a task definition (the search space), LLaMEA iteratively generates, mutates and selects algorithms.<n>We show how this framework can be used to generate novel black-box metaheuristic optimization algorithms automatically.
  arXiv  Detail & Related papers  (2024-05-30T15:10:59Z)
• $\mathbf{(N,K)}$-Puzzle: A Cost-Efficient Testbed for Benchmarking Reinforcement Learning Algorithms in Generative Language Model [50.636423457653066]
  We present a generalized version of the 24-Puzzle: the $(N,K)$-Puzzle, which challenges language models to reach a target value $K$ with $N$ integers. We evaluate the effectiveness of established RL algorithms such as Proximal Policy Optimization (PPO), alongside novel approaches like Identity Policy Optimization (IPO) and Direct Policy Optimization (DPO)
  arXiv  Detail & Related papers  (2024-03-11T22:24:14Z)
• Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers [108.72225067368592]
  We propose a novel perspective to investigate the design of large language models (LLMs)-based prompts.<n>We identify two pivotal factors in model parameter learning: update direction and update method.<n>We develop a capable Gradient-inspired Prompt-based GPO.
  arXiv  Detail & Related papers  (2024-02-27T15:05:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.