Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Models

• URL: http://arxiv.org/abs/2603.28416v1
• Date: Mon, 30 Mar 2026 13:28:01 GMT
• Title: Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Models
• Abstract summary: Reinforcement learning algorithms are defined by their learning update rules, which are typically hand-designed and fixed.<n>We present an evolutionary framework for discovering reinforcement learning algorithms by searching directly over executable update rules that implement complete training procedures.
• Score: 3.890720884484336
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning algorithms are defined by their learning update rules, which are typically hand-designed and fixed. We present an evolutionary framework for discovering reinforcement learning algorithms by searching directly over executable update rules that implement complete training procedures. The approach builds on REvolve, an evolutionary system that uses large language models as generative variation operators, and extends it from reward-function discovery to algorithm discovery. To promote the emergence of nonstandard learning rules, the search excludes canonical mechanisms such as actor--critic structures, temporal-difference losses, and value bootstrapping. Because reinforcement learning algorithms are highly sensitive to internal scalar parameters, we introduce a post-evolution refinement stage in which a large language model proposes feasible hyperparameter ranges for each evolved update rule. Evaluated end-to-end by full training runs on multiple Gymnasium benchmarks, the discovered algorithms achieve competitive performance relative to established baselines, including SAC, PPO, DQN, and A2C.

Related papers

• From AI Assistant to AI Scientist: Autonomous Discovery of LLM-RL Algorithms with LLM Agents [51.12380889298618]
  POISE is a framework for automated discovery of policy optimization algorithms for language models.<n>PoISE maintains a structured, genealogically linked archive linking proposals, executable implementations, standardized evaluations, and natural-language reflections.
  arXiv  Detail & Related papers  (2026-03-25T05:19:23Z)
• Discovering Multiagent Learning Algorithms with Large Language Models [8.649235365712004]
  We propose the use of AlphaEvolve, an evolutionary coding agent powered by large language models, to automatically discover new multiagent learning algorithms.<n>We demonstrate the generality of this framework by evolving novel variants for two distinct paradigms of game-theoretic learning.
  arXiv  Detail & Related papers  (2026-02-18T22:41:00Z)
• Evolutionary Strategies lead to Catastrophic Forgetting in LLMs [51.91763220981834]
  Evolutionary Strategies (ES) have recently re-emerged as a gradient-free alternative to traditional learning algorithms.<n>ES is able to reach performance numbers close to GRPO for math and reasoning tasks with a comparable compute budget.<n>ES is accompanied by significant forgetting of prior abilities, limiting its applicability for training models online.
  arXiv  Detail & Related papers  (2026-01-28T18:59:34Z)
• How Should We Meta-Learn Reinforcement Learning Algorithms? [69.12853522797188]
  We carry out an empirical comparison of the different approaches when applied to a range of meta-learned algorithms.<n>In addition to meta-train and meta-test performance, we also investigate factors including the interpretability, sample cost and train time.<n>We propose several guidelines for meta-learning new RL algorithms which will help ensure that future learned algorithms are as performant as possible.
  arXiv  Detail & Related papers  (2025-07-23T16:31:38Z)
• Discovering Temporally-Aware Reinforcement Learning Algorithms [42.016150906831776]
  We propose a simple augmentation to two existing objective discovery approaches. We find that commonly used meta-gradient approaches fail to discover adaptive objective functions.
  arXiv  Detail & Related papers  (2024-02-08T17:07:42Z)
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [65.57123249246358]
  This paper introduces a pre-trained model-based continual learning toolbox known as PILOT.<n>On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt.<n>On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
  arXiv  Detail & Related papers  (2023-09-13T17:55:11Z)
• In-context Reinforcement Learning with Algorithm Distillation [35.72003039559626]
  We propose a method for distilling reinforcement learning algorithms into neural networks by modeling their training histories with a causal sequence model. A dataset of learning histories is generated by a source RL algorithm, and then a causal transformer is trained by autoregressively predicting actions given their preceding learning histories as context. We demonstrate that AD can learn in-context in a variety of environments with sparse rewards, task structure, and pixel-based observations, and find that AD learns a more data-efficient RL algorithm than the one that generated the source data.
  arXiv  Detail & Related papers  (2022-10-25T17:57:49Z)
• Evolving Reinforcement Learning Algorithms [186.62294652057062]
  We propose a method for meta-learning reinforcement learning algorithms. The learned algorithms are domain-agnostic and can generalize to new environments not seen during training. We highlight two learned algorithms which obtain good generalization performance over other classical control tasks, gridworld type tasks, and Atari games.
  arXiv  Detail & Related papers  (2021-01-08T18:55:07Z)
• Discovering Reinforcement Learning Algorithms [53.72358280495428]
  Reinforcement learning algorithms update an agent's parameters according to one of several possible rules. This paper introduces a new meta-learning approach that discovers an entire update rule. It includes both 'what to predict' (e.g. value functions) and 'how to learn from it' by interacting with a set of environments.
  arXiv  Detail & Related papers  (2020-07-17T07:38:39Z)
• Unbiased Deep Reinforcement Learning: A General Training Framework for Existing and Future Algorithms [3.7050607140679026]
  We propose a novel training framework that is conceptually comprehensible and potentially easy to be generalized to all feasible algorithms for reinforcement learning. We employ Monte-carlo sampling to achieve raw data inputs, and train them in batch to achieve Markov decision process sequences. We propose several algorithms embedded with our new framework to deal with typical discrete and continuous scenarios.
  arXiv  Detail & Related papers  (2020-05-12T01:51:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.