Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training

• URL: http://arxiv.org/abs/2605.07063v1
• Date: Fri, 08 May 2026 00:16:11 GMT
• Title: Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training
• Abstract summary: Dr. Post-Training is a novel framework that reconceptualizes general training data as a data-induced regularizer.<n>We show that our methods consistently outperform state-of-the-art data selection baselines.
• Score: 9.040054119267344
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data selection methods address a critical challenge in LLM post-training: effectively leveraging scarce, high-fidelity target data alongside abundant but imperfectly aligned general training data. In this work, we move beyond the data-selection framing and introduce Dr. Post-Training (Data-Regularized Post-Training), a novel framework that reconceptualizes general training data as a data-induced regularizer that prevents overfitting to the scarce target objective, rather than serving as a pool for selection. Specifically, our framework proposes that at each training step, construct a feasible set of model update directions using the general training data, and project the model update direction specified by the scarce target data onto that feasible set. Standard training and existing data selection methods arise as special cases with different choices of the data-induced regularizer, and these methods correspond to different points on a bias--variance spectrum with different regularization strength. Building on this view, we propose a family of methods offering a richer design space and more flexible bias--variance tradeoffs. For practical LLM-scale use, we introduce careful system optimizations that realize these methods with minimal overhead. Extensive experiments across SFT, RLHF, and RLVR show that our methods consistently outperform state-of-the-art data selection baselines, and system benchmarks confirm their efficiency.

Related papers

• Demystifying Data Organization for Enhanced LLM Training [27.639372382009725]
  Large Language Models (LLMs) have revolutionized various fields, yet their training efficiency is heavily reliant on effective data curation.<n>This paper explores the influence of data organization on LLM training by reusing pre-computed sample-level scores originally generated for data efficiency.
  arXiv  Detail & Related papers  (2026-05-28T17:58:53Z)
• Beyond What to Select: A Plug-and-play Oscillatory Data-Volume Scheduling for Efficient Model Training [36.289092974454796]
  We show that selected-data training induces an implicit regularization effect by the instantaneous selection ratio.<n>This reveals a key trade-off: lower amplify selection-induced regularization, whereas higher ratios preserve data coverage.<n> PODS is compatible with existing static and dynamic selection methods and broadly applicable across training paradigms.
  arXiv  Detail & Related papers  (2026-05-14T12:37:11Z)
• Data Mixing for Large Language Models Pretraining: A Survey and Outlook [52.66761353708619]
  Large language models (LLMs) rely on pretraining on massive and heterogeneous corpora.<n>Unlike sample-level data selection, data mixing optimize domain-level sampling weights to allocate limited budgets more effectively.<n>This paper provides a comprehensive review of data mixing for LLM pretraining.
  arXiv  Detail & Related papers  (2026-03-25T13:30:40Z)
• Sequential Data Augmentation for Generative Recommendation [54.765568804267645]
  Generative recommendation plays a crucial role in personalized systems, predicting users' future interactions from their historical behavior sequences.<n>Data augmentation, the process of constructing training data from user interaction histories, is a critical yet underexplored factor in training these models.<n>We propose GenPAS, a principled framework that models augmentation as a sampling process and enables flexible control of the resulting training distribution.<n>Our experiments on benchmark and industrial datasets demonstrate that GenPAS yields superior accuracy, data efficiency, and parameter efficiency compared to existing strategies.
  arXiv  Detail & Related papers  (2025-09-17T02:53:25Z)
• RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment [10.284993431741377]
  We introduce the concept of epsilon-sample cover, which quantifies sample redundancy based on inter-sample relationships.<n>We reformulate data selection as a reinforcement learning process and propose RL-Selector.<n>Our method consistently outperforms existing state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-06-26T06:28:56Z)
• Ensembles of Low-Rank Expert Adapters [9.599957499802446]
  We propose the Ensembles of Low-Rank Expert Adapters (ELREA) framework to improve the model's capability to handle diverse tasks.<n>ELREA clusters the training instructions based on their gradient directions, representing different areas of expertise.<n>During inference, ELREA combines predictions from the most relevant expert adapters based on the input data's gradient similarity to the training clusters.
  arXiv  Detail & Related papers  (2025-01-31T18:07:21Z)
• Aligning Instruction Tuning with Pre-training [61.50161961371844]
  We propose Aligning Instruction Tuning with Pre-training (AITP) to align instruction tuning with pre-training distributions.<n>We show consistent performance improvements with AITP on three fully open large language models (LLMs) across eight benchmarks.
  arXiv  Detail & Related papers  (2025-01-16T08:27:40Z)
• Capturing the Temporal Dependence of Training Data Influence [100.91355498124527]
  We formalize the concept of trajectory-specific leave-one-out influence, which quantifies the impact of removing a data point during training.<n>We propose data value embedding, a novel technique enabling efficient approximation of trajectory-specific LOO.<n>As data value embedding captures training data ordering, it offers valuable insights into model training dynamics.
  arXiv  Detail & Related papers  (2024-12-12T18:28:55Z)
• ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning [26.069986067128912]
  We introduce ROSE, a Reward-Oriented inStruction data sElection method to optimize data selection for task-specific instruction tuning.<n> ROSE adapts an influence formulation to approximate the influence of training data points relative to a few-shot preference validation set to select the most task-related training data points.
  arXiv  Detail & Related papers  (2024-12-01T01:01:09Z)
• How to Train Data-Efficient LLMs [56.41105687693619]
  We study data-efficient approaches for pre-training language models (LLMs) We find that Ask-LLM and Density sampling are the best methods in their respective categories. In our comparison of 19 samplers, involving hundreds of evaluation tasks and pre-training runs, we find that Ask-LLM and Density are the best methods in their respective categories.
  arXiv  Detail & Related papers  (2024-02-15T02:27:57Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.