DataMaster: Data-Centric Autonomous AI Research

• URL: http://arxiv.org/abs/2605.10906v2
• Date: Wed, 13 May 2026 04:12:44 GMT
• Title: DataMaster: Data-Centric Autonomous AI Research
• Abstract summary: We study task-conditioned autonomous data engineering, where an agent improves a fixed learning algorithm by optimizing only the data side.<n>We propose DataMaster, a data-agent framework that integrates tree-structured search, shared candidate data, and cumulative memory.<n>We evaluate DataMaster on two types of benchmarks, MLE-Bench Lite and PostTrainBench.
• Score: 51.95819363392062
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As model families, training recipes, and compute budgets become increasingly standardized, further gains in machine learning systems depend increasingly on data. Yet data engineering remains largely manual and ad hoc: practitioners repeatedly search for external datasets, adapt them to existing pipelines, validate candidate data through downstream training, and carry forward lessons from prior attempts. We study task-conditioned autonomous data engineering, where an autonomous agent improves a fixed learning algorithm by optimizing only the data side, including external data discovery, data selection and composition, cleaning and transformation. The goal is to obtain a stronger downstream solution while leaving the learning algorithm unchanged. To address the open-ended search space, branch-dependent refinement, and delayed validation inherent in autonomous data engineering, we propose DataMaster, a data-agent framework that integrates tree-structured search, shared candidate data, and cumulative memory. DataMaster consists of three key components: a DataTree that organizes alternative data-engineering branches, a shared Data Pool that stores discovered external data sources for reuse, and a Global Memory that records node outcomes, artifacts, and reusable findings. Together, these components allow the agent to discover candidate data, construct executable training inputs, evaluate them through downstream feedback, and carry useful evidence across branches. We evaluate DataMaster on two types of benchmarks, MLE-Bench Lite and PostTrainBench. On MLE-Bench Lite, it improves medal rate by 32.27% over the initial score; on PostTrainBench, it surpasses the instruct model on GPQA (31.02% vs 30.35%).

Related papers

• DataPrep-Bench: Benchmarking LLMs as Training Data Preparators [35.76000147244654]
  We introduce DataPrep-Bench, the first unified benchmark that jointly evaluates data construction and data quality evaluation.<n>For data construction, methods consume identical raw sources and are scored by fine-tuning a base model on their outputs jointly with Dolly-15k.<n>For data quality evaluation, scoring functions are scored by Pearson correlation with downstream performance on a shared candidate pool.
  arXiv  Detail & Related papers  (2026-05-19T02:23:53Z)
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value [74.80873109856563]
  OpenDataArena (ODA) is a holistic and open platform designed to benchmark the intrinsic value of post-training data.<n>ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; and (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources.
  arXiv  Detail & Related papers  (2025-12-16T03:33:24Z)
• Hierarchical Dataset Selection for High-Quality Data Sharing [6.079330426909266]
  We propose a dataset selection method that models utility at both dataset and group (e.g., collections, institutions) levels.<n>DaSH outperforms state-of-the-art data selection baselines by up to 26.2% in accuracy, while requiring significantly fewer exploration steps.
  arXiv  Detail & Related papers  (2025-12-11T18:59:55Z)
• Dataforge: A Data Agent Platform for Autonomous Data Engineering [22.691284342164334]
  Data Agent is a fully autonomous system specialized for tabular data.<n>It automatically performs data cleaning, hierarchical routing, and feature-level optimization through dual feedback loops.<n>It embodies three core principles: automatic, safe, and non-expert friendly, which ensure end-to-end reliability without human supervision.
  arXiv  Detail & Related papers  (2025-11-09T01:58:13Z)
• Scaling Generalist Data-Analytic Agents [95.05161133349242]
  DataMind is a scalable data synthesis and agent training recipe designed to build generalist data-analytic agents.<n>DataMind tackles three key challenges in building open-source data-analytic agents.
  arXiv  Detail & Related papers  (2025-09-29T17:23:08Z)
• A collaborative digital twin built on FAIR data and compute infrastructure [41.94295877935867]
  This work presents a distributed SDL implementation built on nanoHUB services for online simulation and FAIR data management.<n>Researchers and students can set up their own experiments, share data with collaborators, and explore the combination of FAIR data, predictive ML models, and sequential optimization.
  arXiv  Detail & Related papers  (2025-06-24T18:13:52Z)
• OpenDataLab: Empowering General Artificial Intelligence with Open Datasets [53.22840149601411]
  This paper introduces OpenDataLab, a platform designed to bridge the gap between diverse data sources and the need for unified data processing. OpenDataLab integrates a wide range of open-source AI datasets and enhances data acquisition efficiency through intelligent querying and high-speed downloading services. We anticipate that OpenDataLab will significantly boost artificial general intelligence (AGI) research and facilitate advancements in related AI fields.
  arXiv  Detail & Related papers  (2024-06-04T10:42:01Z)
• An Integrated Data Processing Framework for Pretraining Foundation Models [57.47845148721817]
  Researchers and practitioners often have to manually curate datasets from difference sources. We propose a data processing framework that integrates a Processing Module and an Analyzing Module. The proposed framework is easy to use and highly flexible.
  arXiv  Detail & Related papers  (2024-02-26T07:22:51Z)
• METAM: Goal-Oriented Data Discovery [9.73435089036831]
  METAM is a goal-oriented framework that queries the downstream task with a candidate dataset, forming a feedback loop that automatically steers the discovery and augmentation process. We show METAM's theoretical guarantees and demonstrate those empirically on a broad set of tasks.
  arXiv  Detail & Related papers  (2023-04-18T15:42:25Z)
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier [58.979104709647295]
  We bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2019-12-27T02:05:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.