Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent

• URL: http://arxiv.org/abs/2608.10579v1
• Date: Tue, 11 Aug 2026 07:05:05 GMT
• Title: Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent
• Abstract summary: In this paper, we propose a paradigm shift to automated orchestration via the Instruction Data Selection Agent (DataMaster)<n>By allowing users to specify data needs through natural language descriptions, DataMaster simplifies data curation and removes the burden of manual strategy design.<n>Experiments across the math, medical, and code domains show that DataMaster outperforms static baselines in most settings and surpasses full-pool training in a substantial number of cases.
• Score: 14.370745602969604
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although existing instruction data selection methods have introduced various metrics, the inherent complexity of real-world datasets makes it impractical for any single metric to generalize across all scenarios. Developers are thus often forced to manually inspect data and craft heuristic rules for each new application---a tedious and error-prone process. In this paper, we propose a paradigm shift from manual configuration to automated orchestration via the Instruction Data Selection Agent (DataMaster), which interprets user intent and autonomously composes optimal selection strategies. By allowing users to specify data needs through natural language descriptions, DataMaster simplifies data curation and removes the burden of manual strategy design. Extensive experiments across the math, medical, and code domains show that DataMaster outperforms static baselines in most settings and surpasses full-pool training in a substantial number of cases. The implementation of DataMaster and the scripts needed to reproduce the reported pipeline are publicly available at https://github.com/nju-websoft/DataMaster.

Related papers

• Neurodata Without Boredom: Benchmarking Agentic AI for Data Reuse [0.1494633072572191]
  Neuroscience data are highly fragmented across labs, formats, and experimental paradigms.<n>Formats flexible enough to accommodate diverse experiments are rarely enough to be self-explanatory.<n>General-purpose coding agents performed well on each sub-task, but rarely strung together a fully error-free end-to-end solution.
  arXiv  Detail & Related papers  (2026-05-12T23:00:18Z)
• DataMaster: Data-Centric Autonomous AI Research [51.95819363392062]
  We study task-conditioned autonomous data engineering, where an agent improves a fixed learning algorithm by optimizing only the data side.<n>We propose DataMaster, a data-agent framework that integrates tree-structured search, shared candidate data, and cumulative memory.<n>We evaluate DataMaster on two types of benchmarks, MLE-Bench Lite and PostTrainBench.
  arXiv  Detail & Related papers  (2026-05-11T17:46:24Z)
• TaP: A Taxonomy-Guided Framework for Automated and Scalable Preference Data Generation [50.319535974012]
  Conducting supervised fine-tuning and preference fine-tuning on large language models (LLMs) requires high-quality datasets.<n>Most available datasets for supervised and preference fine-tuning are in English.<n>We propose the underlinetextbfTaxonomy-Guided underlinetextbfPreference Data Generation framework.
  arXiv  Detail & Related papers  (2025-06-30T15:45:28Z)
• Mastering Collaborative Multi-modal Data Selection: A Focus on Informativeness, Uniqueness, and Representativeness [63.484378941471114]
  We propose a collaborative framework, DataTailor, which leverages three key principles--informativeness, uniqueness, and representativeness--for effective data selection.<n>Experiments on various benchmarks demonstrate that DataTailor achieves 101.3% of the performance of full-data fine-tuning with only 15% of the data.
  arXiv  Detail & Related papers  (2024-12-09T08:36:10Z)
• GenQA: Generating Millions of Instructions from a Handful of Prompts [67.54980063851605]
  Most public instruction finetuning datasets are relatively small compared to the closed source datasets used to train industry models. In this work, we study methods for generating large instruction datasets from a single prompt. Our dataset meets or exceeds both WizardLM and Ultrachat on both knowledge-intensive leaderboard tasks as well as conversational evaluations.
  arXiv  Detail & Related papers  (2024-06-14T17:44:08Z)
• Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach [36.47860223750303]
  We consider the problem of automatic curation of high-quality datasets for self-supervised pre-training. We propose a clustering-based approach for building ones satisfying all these criteria. Our method involves successive and hierarchical applications of $k$-means on a large and diverse data repository.
  arXiv  Detail & Related papers  (2024-05-24T14:58:51Z)
• UniDM: A Unified Framework for Data Manipulation with Large Language Models [66.61466011795798]
  Large Language Models (LLMs) resolve multiple data manipulation tasks. LLMs exhibit bright benefits in terms of performance but still require customized designs to fit each specific task. We propose UniDM, a unified framework which establishes a new paradigm to process data manipulation tasks.
  arXiv  Detail & Related papers  (2024-05-10T14:44:04Z)
• Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets [73.2096288987301]
  We propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset. We observe that our method learns to query only the relevant transitions to the task, filtering out sub-optimal or task-irrelevant data. Our simple querying approach outperforms more complex goal-conditioned methods by 20% across simulated and real robotic manipulation tasks from images.
  arXiv  Detail & Related papers  (2023-04-18T05:42:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.