AlphaLab: Autonomous Multi-Agent Research Across Optimization Domains with Frontier LLMs

• URL: http://arxiv.org/abs/2604.08590v1
• Date: Tue, 31 Mar 2026 21:16:20 GMT
• Title: AlphaLab: Autonomous Multi-Agent Research Across Optimization Domains with Frontier LLMs
• Abstract summary: We present AlphaLab, an autonomous research harness that automates the full experimental cycle in computation-intensive domains.<n>It adapts to the domain and explores the data, writing analysis code and producing a research report.<n>It runs large-scale experiments via a Strategist/Worker loop, accumulating domain knowledge in a persistent playbook.
• Score: 14.167136550376497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present AlphaLab, an autonomous research harness that leverages frontier LLM agentic capabilities to automate the full experimental cycle in quantitative, computation-intensive domains. Given only a dataset and a natural-language objective, AlphaLab proceeds through three phases without human intervention: (1) it adapts to the domain and explores the data, writing analysis code and producing a research report; (2) it constructs and adversarially validates its own evaluation framework; and (3) it runs large-scale GPU experiments via a Strategist/Worker loop, accumulating domain knowledge in a persistent playbook that functions as a form of online prompt optimization. All domain-specific behavior is factored into adapters generated by the model itself, so the same pipeline handles qualitatively different tasks without modification. We evaluate AlphaLab with two frontier LLMs (GPT-5.2 and Claude Opus 4.6) on three domains: CUDA kernel optimization, where it writes GPU kernels that run 4.4x faster than torch.compile on average (up to 91x); LLM pretraining, where the full system achieves 22% lower validation loss than a single-shot baseline using the same model; and traffic forecasting, where it beats standard baselines by 23-25% after researching and implementing published model families from the literature. The two models discover qualitatively different solutions in every domain (neither dominates uniformly), suggesting that multi-model campaigns provide complementary search coverage. We additionally report results on financial time series forecasting in the appendix, and release all code at https://brendanhogan.github.io/alphalab-paper/.

Related papers

• A Dataset for Modeling Iterative Problem-Solving [26.98102675986399]
  We build a benchmark that evaluates models spanning parametric, sequential, and generative traditions.<n>We find that the model's coding proficiency is inversely related to predictive performance in this setting.
  arXiv  Detail & Related papers  (2026-09-01T09:00:55Z)
• optimize_anything: A Universal API for Optimizing any Text Parameter [98.42497715725356]
  We show a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs.<n>We show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm.
  arXiv  Detail & Related papers  (2026-05-19T10:18:12Z)
• Deep Researcher Agent: An Autonomous Framework for 24/7 Deep Learning Experimentation with Zero-Cost Monitoring [2.4112990554464235]
  We present textbfDeep Researcher Agent, an open-source framework that enables large language model (LLM) agents to autonomously conduct deep learning experiments around the clock.<n>Unlike existing AI research assistants that focus on paper writing or code generation, our system addresses the full experiment lifecycle: hypothesis formation, code implementation, training execution, result analysis, and iterative refinement.
  arXiv  Detail & Related papers  (2026-04-07T13:16:31Z)
• RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets [0.2126018197183873]
  We present the first systematic comparison of three optimization paradigms -- Elo tournament selection, GEPA, and greedy hill-climbing.<n>RoboPhD introduces validation-free evolution: instead of splitting the budget between training and validation, it uses Elo competition on training data to simultaneously evaluate agents and drive evolution.<n>We release RoboPhD as a versatile toolkit under the MIT license with a simple optimize_anything() API for evolving diverse complex agents.
  arXiv  Detail & Related papers  (2026-04-06T01:39:13Z)
• Evaluating Efficiency and Novelty of LLM-Generated Code for Graph Analysis [0.1274452325287335]
  Large Language Models (LLMs) are increasingly used to automate software development.<n>We present a comprehensive study of LLMs' ability to generate efficient C implementations of graph-analysis routines.<n>The potential for these models to invent transformative algorithmic techniques represents a compelling frontier for future research.
  arXiv  Detail & Related papers  (2025-07-09T00:46:30Z)
• APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay [86.01901238059261]
  APIGen-MT is a framework that generates verifiable and diverse multi-turn agent data.<n>We train a family of models -- the xLAM-2-fc-r series with sizes ranging from 1B to 70B parameters.<n>Our models outperform frontier models such as GPT-4o and Claude 3.5 on $tau$-bench and BFCL benchmarks.
  arXiv  Detail & Related papers  (2025-04-04T17:13:57Z)
• AIvaluateXR: An Evaluation Framework for on-Device AI in XR with Benchmarking Results [55.33807002543901]
  We present AIvaluateXR, a comprehensive evaluation framework for benchmarking large language models (LLMs) running on XR devices.<n>We deploy 17 selected LLMs across four XR platforms: Magic Leap 2, Meta Quest 3, Vivo X100s Pro, and Apple Vision Pro, and conduct an extensive evaluation.<n>We propose a unified evaluation method based on the 3D Optimality theory to select the optimal device-model pairs from quality and speed objectives.
  arXiv  Detail & Related papers  (2025-02-13T20:55:48Z)
• Reinforcement Learning for Long-Horizon Interactive LLM Agents [56.9860859585028]
  Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests.<n>We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments.<n>We derive LOOP, a data- and memory-efficient variant of proximal policy optimization.
  arXiv  Detail & Related papers  (2025-02-03T18:35:42Z)
• Exploring Distributional Shifts in Large Language Models for Code Analysis [36.73114441988879]
  We study how three large language models with code capabilities generalize to out-of-domain data. We consider two fundamental applications - code summarization, and code generation. We find that a model adapted to multiple domains simultaneously performs on par with those adapted to a single domain.
  arXiv  Detail & Related papers  (2023-03-16T07:45:46Z)
• Learning To Dive In Branch And Bound [95.13209326119153]
  We propose L2Dive to learn specific diving structurals with graph neural networks. We train generative models to predict variable assignments and leverage the duality of linear programs to make diving decisions.
  arXiv  Detail & Related papers  (2023-01-24T12:01:45Z)
• Gradient Matching for Domain Generalization [93.04545793814486]
  A critical requirement of machine learning systems is their ability to generalize to unseen domains. We propose an inter-domain gradient matching objective that targets domain generalization. We derive a simpler first-order algorithm named Fish that approximates its optimization.
  arXiv  Detail & Related papers  (2021-04-20T12:55:37Z)
• Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation [87.53808756910452]
  We propose a novel, flexible and accurate refinement module called Alpha-Refine. It exploits a precise pixel-wise correlation layer together with a spatial-aware non-local layer to fuse features and can predict three complementary outputs: bounding box, corners and mask. We apply the proposed Alpha-Refine module to five famous and state-of-the-art base trackers: DiMP, ATOM, SiamRPN++, RTMDNet and ECO.
  arXiv  Detail & Related papers  (2020-07-04T07:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.