Procedural Generation of Algorithm Discovery Tasks in Machine Learning

• URL: http://arxiv.org/abs/2603.17863v1
• Date: Wed, 18 Mar 2026 15:49:32 GMT
• Title: Procedural Generation of Algorithm Discovery Tasks in Machine Learning
• Abstract summary: DiscoGen is a procedural generator of algorithm discovery tasks for machine learning.<n>It spans millions of tasks of varying difficulty and complexity from a range of machine learning fields.<n>We present a benchmark consisting of a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs.
• Score: 87.89048339690163
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans millions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise algorithm discovery agents (ADAs). We present DiscoBench, a benchmark consisting of a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, in addition to experiments demonstrating its use for prompt optimisation of an ADA. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen.

Related papers

• RL4RLA: Teaching ML to Discover Randomized Linear Algebra Algorithms Through Curriculum Design and Graph-Based Search [18.388661543349]
  We present RL4RLA, a general RL framework that automates the discovery of interpretable, symbolic RLA algorithms.<n>Unlike black-box approaches, our method builds explicit algorithms from basic linear algebra primitives, ensuring verifiable and implementable representations.<n>We demonstrate that RL4RLA rediscovers state-of-the-art methods, including sketch-and-precondition solvers, Randomized Kaczmarz, and Newton Sketch.
  arXiv  Detail & Related papers  (2026-05-18T07:57:26Z)
• SciML Agents: Write the Solver, Not the Solution [69.5021018644143]
  We introduce two new datasets: a diagnostic dataset of adversarial "misleading" problems; and a large-scale benchmark of 1,000 diverse ODE tasks.<n>We evaluate open- and closed-source LLM models along two axes: (i) unguided versus guided prompting with domain-specific knowledge; and (ii) off-the-shelf versus fine-tuned variants.<n>Preliminary results indicate that careful prompting and fine-tuning can yield a specialized LLM agent capable of reliably solving simple ODE problems.
  arXiv  Detail & Related papers  (2025-09-12T02:53:57Z)
• Generative Modeling for Mathematical Discovery [0.19791587637442667]
  We present a new implementation of the genetic algorithm it funsearch.<n>Our aim is to generate examples of interest to mathematicians.<n>It does not require expertise in machine learning or access to high-performance computing resources.
  arXiv  Detail & Related papers  (2025-03-14T03:54:43Z)
• The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks [59.26515696183751]
  We show that algorithm discovery in neural networks is sometimes more complex. We show that even simple learning problems can admit a surprising diversity of solutions.
  arXiv  Detail & Related papers  (2023-06-30T17:59:13Z)
• Generative and reproducible benchmarks for comprehensive evaluation of machine learning classifiers [6.605210393590192]
  DIverse and GENerative ML Benchmark (DIGEN) is a collection of synthetic datasets for benchmarking of machine learning algorithms. The resource with extensive documentation and analyses is open-source and available on GitHub.
  arXiv  Detail & Related papers  (2021-07-14T03:58:02Z)
• Evolving Reinforcement Learning Algorithms [186.62294652057062]
  We propose a method for meta-learning reinforcement learning algorithms. The learned algorithms are domain-agnostic and can generalize to new environments not seen during training. We highlight two learned algorithms which obtain good generalization performance over other classical control tasks, gridworld type tasks, and Atari games.
  arXiv  Detail & Related papers  (2021-01-08T18:55:07Z)
• Discovering Reinforcement Learning Algorithms [53.72358280495428]
  Reinforcement learning algorithms update an agent's parameters according to one of several possible rules. This paper introduces a new meta-learning approach that discovers an entire update rule. It includes both 'what to predict' (e.g. value functions) and 'how to learn from it' by interacting with a set of environments.
  arXiv  Detail & Related papers  (2020-07-17T07:38:39Z)
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch [76.83052807776276]
  We show that it is possible to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space. We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction in the field.
  arXiv  Detail & Related papers  (2020-03-06T19:00:04Z)
• Neuroevolution of Neural Network Architectures Using CoDeepNEAT and Keras [0.0]
  A large portion of the work involved in a machine learning project is to define the best type of algorithm to solve a given problem. Finding the optimal network topology and configurations for a given problem is a challenge that requires domain knowledge and testing efforts.
  arXiv  Detail & Related papers  (2020-02-11T19:03:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.