Generative Data Refinement: Just Ask for Better Data

• URL: http://arxiv.org/abs/2509.08653v2
• Date: Thu, 11 Sep 2025 11:18:17 GMT
• Title: Generative Data Refinement: Just Ask for Better Data
• Authors: Minqi Jiang, João G. M. Araújo, Will Ellsworth, Sian Gooding, Edward Grefenstette,
• Abstract summary: Training datasets now grow faster than the rate at which new data is indexed on the web.<n>Much more data exists as user-generated content that is not publicly indexed, but incorporating such data comes with considerable risks.<n>We introduce a framework, Generative Data Refinement (GDR), for using pretrained generative models to transform a dataset with undesirable content into a refined dataset.
• Score: 19.774236070314963
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: For a fixed parameter size, the capabilities of large models are primarily determined by the quality and quantity of its training data. Consequently, training datasets now grow faster than the rate at which new data is indexed on the web, leading to projected data exhaustion over the next decade. Much more data exists as user-generated content that is not publicly indexed, but incorporating such data comes with considerable risks, such as leaking private information and other undesirable content. We introduce a framework, Generative Data Refinement (GDR), for using pretrained generative models to transform a dataset with undesirable content into a refined dataset that is more suitable for training. Our experiments show that GDR can outperform industry-grade solutions for dataset anonymization, as well as enable direct detoxification of highly unsafe datasets. Moreover, we show that by generating synthetic data that is conditioned on each example in the real dataset, GDR's refined outputs naturally match the diversity of web scale datasets, and thereby avoid the often challenging task of generating diverse synthetic data via model prompting. The simplicity and effectiveness of GDR make it a powerful tool for scaling up the total stock of training data for frontier models.

Related papers

• Towards Robust In-Context Learning for Medical Image Segmentation via Data Synthesis [12.827413884809644]
  In-Context Learning (ICL) for universal medical image segmentation has introduced an unprecedented demand for large-scale, diverse datasets for training.<n>We propose textbf SynthICL, a novel data synthesis framework built upon domain randomization.<n>Our work helps mitigate the data bottleneck for ICL-based segmentation, paving the way for robust models.
  arXiv  Detail & Related papers  (2025-09-24T02:44:53Z)
• DataMIL: Selecting Data for Robot Imitation Learning with Datamodels [77.48472034791213]
  We introduce DataMIL, a policy-driven data selection framework built on the datamodels paradigm.<n>Unlike standard practices that filter data using human notions of quality, DataMIL directly optimize data selection for task success.<n>We validate our approach on a suite of more than 60 simulation and real-world manipulation tasks.
  arXiv  Detail & Related papers  (2025-05-14T17:55:10Z)
• DataGen: Unified Synthetic Dataset Generation via Large Language Models [88.16197692794707]
  DataGen is a comprehensive framework designed to produce diverse, accurate, and highly controllable datasets.<n>To augment data diversity, DataGen incorporates an attribute-guided generation module and a group checking feature.<n>Extensive experiments demonstrate the superior quality of data generated by DataGen.
  arXiv  Detail & Related papers  (2024-06-27T07:56:44Z)
• MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Data [10.217822818544475]
  We propose a framework to generate synthetic (tabular) data powered by large language models (LLMs)<n>Our approach significantly enhance the quality of synthetic data generation in common scenarios with small sample sizes.<n>Our results demonstrate that our model outperforms several state-of-art models regarding generating higher quality synthetic data for downstream tasks while keeping privacy of the real data.
  arXiv  Detail & Related papers  (2024-06-15T06:26:17Z)
• Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks [66.87070857705994]
  In low-resource settings, the amount of seed data samples to use for data augmentation is very small. We propose a novel method that augments training data by incorporating a wealth of examples from other datasets. This approach can ensure that the generated data is not only relevant but also more diverse than what could be achieved using the limited seed data alone.
  arXiv  Detail & Related papers  (2024-02-21T02:45:46Z)
• Multi-Resolution Diffusion for Privacy-Sensitive Recommender Systems [2.812395851874055]
  We introduce a Score-based Diffusion Recommendation Module (SDRM), which captures the intricate patterns of real-world datasets required for training highly accurate recommender systems. SDRM allows for the generation of synthetic data that can replace existing datasets to preserve user privacy, or augment existing datasets to address excessive data sparsity. Our method outperforms competing baselines such as generative adversarial networks, variational autoencoders, and recently proposed diffusion models in synthesizing various datasets to replace or augment the original data by an average improvement of 4.30% in Recall@k and 4.65% in NDCG@k.
  arXiv  Detail & Related papers  (2023-11-06T19:52:55Z)
• Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark [56.8042116967334]
  Synthetic data serves as an alternative in training machine learning models. ensuring that synthetic data mirrors the complex nuances of real-world data is a challenging task. This paper explores the potential of integrating data-centric AI techniques to guide the synthetic data generation process.
  arXiv  Detail & Related papers  (2023-10-25T20:32:02Z)
• STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models [56.27786433792638]
  STAR is a data generation method that leverages Large Language Models (LLMs) to synthesize data instances. We design fine-grained step-by-step instructions to obtain the initial data instances. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks.
  arXiv  Detail & Related papers  (2023-05-24T12:15:19Z)
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier [58.979104709647295]
  We bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2019-12-27T02:05:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.