Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods

• URL: http://arxiv.org/abs/2605.05227v1
• Date: Sun, 19 Apr 2026 14:23:23 GMT
• Title: Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
• Abstract summary: ADAPT is a dynamic online framework that reweights training samples with adaptive per-sample learning rates guided by similarity-based quality signals.<n> ADAPT consistently outperforms offline selection/mixing and prior online methods, achieving stronger cross-benchmark generalization under equal FLOPs.
• Score: 30.96121293915128
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Data curation is a critical yet under-explored area in large language model (LLM) training. Existing methods, such as data selection and mixing, operate in an offline paradigm, detaching themselves from training. This separation introduces engineering overhead and makes the curation brittle: the entire pipeline must be re-run under model/task shifts. Moreover, offline methods alter data size through hard filtering or resampling, often sacrificing data diversity and harming generalization. We propose to rethink data curation as an online reweighting problem, where sample importance is dynamically adjusted during training via loss weighting rather than static pre-processing. Specifically, we introduce ADAPT (Adaptive Data reweighting for Pretraining and FineTuning), a dynamic online framework that reweights training samples with adaptive per-sample learning rates guided by similarity-based quality signals, without changing the number of training samples. Unlike offline methods that enforce a static data distribution, ADAPT acts as an implicit curriculum learner, progressively shifting focus from coarse-grained patterns to fine-grained semantic distinctions as the model evolves. Experiments on both instruction tuning and large-scale pretraining show that ADAPT consistently outperforms offline selection/mixing and prior online methods, achieving stronger cross-benchmark generalization under equal FLOPs.

Related papers

• STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations [46.963298896594814]
  Training Data Attribution seeks to trace a model's predictions back to its training data.<n>Most approaches approximate this effect in the parameter space using gradients.<n>We model the functional effect of training data in the activation space.
  arXiv  Detail & Related papers  (2026-06-03T17:59:36Z)
• Beyond What to Select: A Plug-and-play Oscillatory Data-Volume Scheduling for Efficient Model Training [36.289092974454796]
  We show that selected-data training induces an implicit regularization effect by the instantaneous selection ratio.<n>This reveals a key trade-off: lower amplify selection-induced regularization, whereas higher ratios preserve data coverage.<n> PODS is compatible with existing static and dynamic selection methods and broadly applicable across training paradigms.
  arXiv  Detail & Related papers  (2026-05-14T12:37:11Z)
• ROAD: Adaptive Data Mixing for Offline-to-Online Reinforcement Learning via Bi-Level Optimization [19.46945184186446]
  We propose Reinforcement Learning with Optimized Adaptive Data-mixing (ROAD)<n>ROAD is a dynamic plug-and-play framework that automates the data replay process.<n>Our empirical results demonstrate that this approach consistently outperforms existing data replay methods across various datasets.
  arXiv  Detail & Related papers  (2026-05-14T07:35:58Z)
• Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks [10.269961654607108]
  Deep neural networks are typically trained by uniformly sampling large datasets across epochs.<n>Recent work shows that progressively reducing the amount of training data can improve efficiency and generalization.<n>We propose Adaptive Data Dropout, a framework that dynamically adjusts the subset of training data based on performance feedback.
  arXiv  Detail & Related papers  (2026-04-14T16:41:33Z)
• DIET: Learning to Distill Dataset Continually for Recommender Systems [69.68271683922536]
  Deep recommender models are trained under a continual learning paradigm, relying on massive and continuously growing streaming behavioral logs.<n>In large-scale platforms, retraining models on full historical data for architecture comparison or iteration is prohibitively expensive, severely slowing down model development.<n>We formulate this problem as emphstreaming dataset distillation for recommender systems and propose textbfDIET, a unified framework that maintains a compact distilled dataset which evolves alongside streaming data while preserving training-critical signals.
  arXiv  Detail & Related papers  (2026-03-26T02:50:13Z)
• ToReMi: Topic-Aware Data Reweighting for Dynamic Pre-Training Data Selection [28.75333303894706]
  ToReMi is a novel framework that adjusts training sample weights according to their topical associations and observed learning patterns.<n>Our experiments reveal that ToReMi variants consistently achieve superior performance over conventional pre-training approaches.
  arXiv  Detail & Related papers  (2025-04-01T12:06:42Z)
• Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining [55.262510814326035]
  Existing reweighting strategies primarily focus on group-level data importance.<n>We introduce novel algorithms for dynamic, instance-level data reweighting.<n>Our framework allows us to devise reweighting strategies deprioritizing redundant or uninformative data.
  arXiv  Detail & Related papers  (2025-02-10T17:57:15Z)
• What Do Learning Dynamics Reveal About Generalization in LLM Reasoning? [83.83230167222852]
  We find that a model's generalization behavior can be effectively characterized by a training metric we call pre-memorization train accuracy. By connecting a model's learning behavior to its generalization, pre-memorization train accuracy can guide targeted improvements to training strategies.
  arXiv  Detail & Related papers  (2024-11-12T09:52:40Z)
• MelissaDL x Breed: Towards Data-Efficient On-line Supervised Training of Multi-parametric Surrogates with Active Learning [0.0]
  We introduce a new active learning method to enhance data-efficiency for on-line surrogate training. The surrogate is trained to predict a given timestep directly with different initial and boundary conditions parameters. Preliminary results for 2D heat PDE demonstrate the potential of this method, called Breed, to improve the generalization capabilities of surrogates.
  arXiv  Detail & Related papers  (2024-10-08T09:52:15Z)
• Distilled Datamodel with Reverse Gradient Matching [74.75248610868685]
  We introduce an efficient framework for assessing data impact, comprising offline training and online evaluation stages. Our proposed method achieves comparable model behavior evaluation while significantly speeding up the process compared to the direct retraining method.
  arXiv  Detail & Related papers  (2024-04-22T09:16:14Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.