Online Data Selection for Instruction Tuning via Gaussian Processes

• URL: http://arxiv.org/abs/2606.30077v1
• Date: Mon, 29 Jun 2026 10:08:12 GMT
• Title: Online Data Selection for Instruction Tuning via Gaussian Processes
• Abstract summary: Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality.<n>Existing online data selection methods for LLM training are typically "batch-constrained"<n>We propose GAIA, a framework that formulates data valuation as a global estimation process.
• Score: 11.553425714780076
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. Existing online data selection methods for LLM training are typically "batch-constrained", limiting optimization to local utility within random batches. To overcome this, we propose GAIA (Global Adaptive Instruction tuning via GAussian processes), a framework that formulates data valuation as a global estimation process. GAIA employs Gaussian Process regression to model continuous utility manifolds across the semantic space, utilizing an adaptive strategy fusion mechanism to dynamically prioritize high-utility samples. By casting the strategy-posterior update as an instance of the classical fixed-share Hedge framework for tracking the best expert, we inherit a dynamic-regret guarantee that characterizes GAIA's robustness under non-stationary quality scores during training. Empirical evaluations on three datasets demonstrate that GAIA significantly outperforms state-of-the-art baselines like \greats, establishing our method as a scalable and robust solution for efficient instruction tuning.

Related papers

• Dynamic Important Example Mining for Reinforcement Finetuning [53.98969373888718]
  Reinforcement fine-tuning (RFT) is increasingly used to strengthen the reasoning abilities of large models.<n>Most data-centric RFT methods rely on static or sample selection, implicitly assuming a sample's value is fixed over training.<n>We propose Dynamic Important Example Mining (DIEM), a principled and fully automated framework that makes data utilization adaptive throughout RFT.
  arXiv  Detail & Related papers  (2026-08-29T13:12:25Z)
• Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training [9.040054119267344]
  Dr. Post-Training is a novel framework that reconceptualizes general training data as a data-induced regularizer.<n>We show that our methods consistently outperform state-of-the-art data selection baselines.
  arXiv  Detail & Related papers  (2026-05-08T00:16:11Z)
• Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation [21.694351921779845]
  Fine-tuning large language models for specialized domains often necessitates a trade-off between acquiring domain expertise and retaining general reasoning capabilities.<n>We propose Orthogonal Gradient Selection (OGS), a data-centric method that harmonizes domain performance, general capability retention, and training efficiency.
  arXiv  Detail & Related papers  (2026-02-06T03:41:40Z)
• Predict the Retrieval! Test time adaptation for Retrieval Augmented Generation [66.36556189794526]
  TTARAG is a test-time adaptation method that dynamically updates the language model's parameters during inference to improve RAG system performance in specialized domains.<n>Our method introduces a simple yet effective approach where the model learns to predict retrieved content, enabling automatic parameter adjustment to the target domain.
  arXiv  Detail & Related papers  (2026-01-16T17:07:01Z)
• Heterogeneous Self-Supervised Acoustic Pre-Training with Local Constraints [64.15709757611369]
  We propose a new self-supervised pre-training approach to dealing with heterogeneous data.<n>The proposed approach can significantly improve the adaptivity of the self-supervised pre-trained model for the downstream supervised fine-tuning tasks.
  arXiv  Detail & Related papers  (2025-08-27T15:48:50Z)
• Improving Model Classification by Optimizing the Training Dataset [3.987352341101438]
  Coresets offer a principled approach to data reduction, enabling efficient learning on large datasets.<n>We present a systematic framework for tuning the coreset generation process to enhance downstream classification quality.
  arXiv  Detail & Related papers  (2025-07-22T16:10:11Z)
• Gradient Correction in Federated Learning with Adaptive Optimization [19.93709245766609]
  We propose tt FAdamGC, the first algorithm to integrate client-drift compensation into adaptive optimization.<n>We show that tt FAdamGC consistently outperforms existing methods in total communication and cost across varying levels of data.
  arXiv  Detail & Related papers  (2025-02-04T21:21:30Z)
• Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment [104.18002641195442]
  We introduce Self-Augmented Preference Optimization (SAPO), an effective and scalable training paradigm that does not require existing paired data. Building on the self-play concept, which autonomously generates negative responses, we further incorporate an off-policy learning pipeline to enhance data exploration and exploitation.
  arXiv  Detail & Related papers  (2024-05-31T14:21:04Z)
• A Conditioned Unsupervised Regression Framework Attuned to the Dynamic Nature of Data Streams [0.0]
  This paper presents an optimal strategy for streaming contexts with limited labeled data, introducing an adaptive technique for unsupervised regression. The proposed method leverages a sparse set of initial labels and introduces an innovative drift detection mechanism. To enhance adaptability, we integrate the ADWIN (ADaptive WINdowing) algorithm with error generalization based on Root Mean Square Error (RMSE)
  arXiv  Detail & Related papers  (2023-12-12T19:23:54Z)
• TRIAGE: Characterizing and auditing training data for improved regression [80.11415390605215]
  We introduce TRIAGE, a novel data characterization framework tailored to regression tasks and compatible with a broad class of regressors. TRIAGE utilizes conformal predictive distributions to provide a model-agnostic scoring method, the TRIAGE score. We show that TRIAGE's characterization is consistent and highlight its utility to improve performance via data sculpting/filtering, in multiple regression settings.
  arXiv  Detail & Related papers  (2023-10-29T10:31:59Z)
• Consistency Regularization for Generalizable Source-free Domain Adaptation [62.654883736925456]
  Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset. Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets. We propose a consistency regularization framework to develop a more generalizable SFDA method.
  arXiv  Detail & Related papers  (2023-08-03T07:45:53Z)
• DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks [88.62288327934499]
  We propose a novel augmentation method with language models trained on the linearized labeled sentences. Our method is applicable to both supervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2020-11-03T07:49:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.