BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training

• URL: http://arxiv.org/abs/2606.18650v1
• Date: Wed, 17 Jun 2026 03:35:04 GMT
• Title: BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training
• Abstract summary: We propose BLADE (Bi-Level Adaptive Data sElection), a Hessian-free framework for data selection.<n> BLADE reformulates the bi-level optimization problem underlying influence-based methods as a penalized single-level objective.<n>For efficient online batch selection, we instantiate BLADE as a memoryless randomized block-coordinate Frank-Wolfe algorithm.
• Score: 34.64488220249771
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories. Beyond static heuristic filtering, advanced data selection methods for LLM training largely follow two paradigms, each with fundamental limitations. Influence-based methods provide principled bi-level objectives but require intractable inverse-Hessian computations, while excess-loss methods are computationally efficient but rely on a static reference model that becomes misaligned with the evolving proxy model during training. We propose BLADE (Bi-Level Adaptive Data sElection), a Hessian-free framework for data selection. BLADE reformulates the bi-level optimization problem underlying influence-based methods as a penalized single-level objective via Lagrange multipliers, avoiding inverse-Hessian computation while revealing a principled connection to excess-loss based data selection. The resulting objective recovers an excess-loss form but replaces the static reference model with a dynamic one that stays synchronized with training. Theoretically, we prove that this penalized formulation guarantees first-order convergence. For efficient online batch selection, we instantiate BLADE as a memoryless randomized block-coordinate Frank-Wolfe algorithm. Extensive experiments show that BLADE consistently outperforms state-of-the-art data selection baselines, providing a practical recipe for LLM training.

Related papers

• BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining [28.32850393150554]
  BLISS is a lightweight data selection method that operates entirely emphfrom scratch, without relying on any external pretrained oracle models.<n>We validate BLISS by pretraining 410M/1B/2.8B Pythia and LLaMA-0.5B models on selected subsets of the C4 dataset.<n> BLISS achieves $1.7times$ speedup in reaching the same performance as the state-of-the-art method.
  arXiv  Detail & Related papers  (2025-10-07T15:42:33Z)
• LLM Routing with Dueling Feedback [49.67815163970033]
  We study the problem of selecting the best model for each query while balancing user satisfaction, model expertise, and inference cost.<n>We formulate routing as contextual dueling bandits, learning from pairwise preference feedback rather than absolute scores.<n>We introduce Category-Calibrated Fine-Tuning (CCFT), a representation-learning method that derives model embeddings from offline data using contrastive fine-tuning with categorical weighting.
  arXiv  Detail & Related papers  (2025-10-01T12:52:25Z)
• LAMDAS: LLM as an Implicit Classifier for Domain-specific Data Selection [32.35731324386828]
  Adapting large language models (LLMs) to specific domains often faces a critical bottleneck: the scarcity of high-quality, human-curated data.<n>Existing approaches, categorized as similarity-based and direct optimization methods, struggle to simultaneously achieve these goals.<n>We introduce LAMDAS, a novel approach that leverages the pre-trained LLM itself as an implicit classifier.
  arXiv  Detail & Related papers  (2025-09-08T10:30:58Z)
• ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment [94.36403843133616]
  Using human preferences to align large language models (LLMs) has significantly improved their performance in various downstream tasks.<n>Existing methods either lack a strong theoretical foundation or depend on restrictive reward function assumptions.<n>We propose an algorithm, ActiveDPO, that uses a theoretically grounded data selection criterion for non-linear reward functions.
  arXiv  Detail & Related papers  (2025-05-25T17:42:52Z)
• Transfer Learning of CATE with Kernel Ridge Regression [4.588222946914528]
  We propose a novel method for overlap-adaptive transfer learning of conditional average treatment effect (CATE) using kernel ridge regression (KRR)<n>We provide a theoretical justification for our method through sharp non-asymptotic MSE bounds, highlighting its adaptivity to both weak overlaps and the complexity of CATE function.
  arXiv  Detail & Related papers  (2025-02-17T01:07:45Z)
• Provably Efficient Online RLHF with One-Pass Reward Modeling [70.82499103200402]
  Reinforcement Learning from Human Feedback has shown remarkable success in aligning Large Language Models with human preferences.<n>Online RLHF has emerged as a promising direction, enabling iterative data collection and refinement.<n>We propose a one-pass reward modeling method that eliminates the need to store historical data and achieves constant-time updates per iteration.
  arXiv  Detail & Related papers  (2025-02-11T02:36:01Z)
• Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning [45.64632177923583]
  This work studies the problem of large language model (LLM) unlearning, aiming to remove unwanted data influences.<n>Despite the increasing demand for unlearning, a technically-grounded optimization framework is lacking.<n>We propose a simple yet effective unlearning optimization framework, called SimNPO, showing that simplicity' in removing the reliance on a reference model benefits unlearning.
  arXiv  Detail & Related papers  (2024-10-09T17:58:12Z)
• Entropy Law: The Story Behind Data Compression and LLM Performance [115.70395740286422]
  We find that model performance is negatively correlated to the compression ratio of training data, which usually yields a lower training loss. Based on the findings of the entropy law, we propose a quite efficient and universal data selection method. We also present an interesting application of entropy law that can detect potential performance risks at the beginning of model training.
  arXiv  Detail & Related papers  (2024-07-09T08:14:29Z)
• Oracle Inequalities for Model Selection in Offline Reinforcement Learning [105.74139523696284]
  We study the problem of model selection in offline RL with value function approximation. We propose the first model selection algorithm for offline RL that achieves minimax rate-optimal inequalities up to logarithmic factors. We conclude with several numerical simulations showing it is capable of reliably selecting a good model class.
  arXiv  Detail & Related papers  (2022-11-03T17:32:34Z)
• A Lagrangian Duality Approach to Active Learning [119.36233726867992]
  We consider the batch active learning problem, where only a subset of the training data is labeled. We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples. We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-02-08T19:18:49Z)
• Bayesian Active Learning with Pretrained Language Models [9.161353418331245]
  Active Learning (AL) is a method to iteratively select data for annotation from a pool of unlabeled data. Previous AL approaches have been limited to task-specific models that are trained from scratch at each iteration. We introduce BALM; Bayesian Active Learning with pretrained language models.
  arXiv  Detail & Related papers  (2021-04-16T19:07:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.