Market-Driven Subset Selection for Budgeted Training

• URL: http://arxiv.org/abs/2510.02456v2
• Date: Mon, 20 Oct 2025 15:38:47 GMT
• Title: Market-Driven Subset Selection for Budgeted Training
• Abstract summary: We propose a market-based framework that treats each training example as a tradeable contract.<n>On GSM8K mathematical reasoning under strict 60k-token budgets, our selector achieves parity with strong single-signal baselines.<n>Our framework unifies multi-signal data curation under fixed computational budgets for prompt-level reasoning and classification tasks.
• Score: 1.7969777786551429
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Training large language models on massive datasets is computationally expensive, yet empirical evidence suggests that substantial portions of training examples contribute minimally to final performance. Data subset selection addresses this inefficiency by identifying small, high-utility subsets under resource constraints. However, example utility is inherently multi-faceted, encompassing uncertainty, distributional rarity, and diversity signals that are heterogeneous and typically combined through ad hoc weighted sums lacking theoretical grounding. We propose a market-based framework that treats each training example as a tradeable contract and employs the Logarithmic Market Scoring Rule to aggregate multiple utility signals into coherent prices. Heterogeneous signals act as traders, a single liquidity parameter controls concentration versus smoothing, and topic-wise normalization ensures calibrated aggregation. Token budgets are handled explicitly through a price-per-token decision rule with an interpretable length-bias parameter. We establish theoretical connections to maximum-entropy aggregation and provide utility recovery guarantees under noisy but monotone signals. On GSM8K mathematical reasoning under strict 60k-token budgets, our selector achieves parity with strong single-signal baselines while exhibiting lower variance and incurring less than 0.1 GPU-hour overhead. On AGNews classification at 5-25\% retention rates, the market formulation delivers competitive accuracy with improved stability. Our framework unifies multi-signal data curation under fixed computational budgets for prompt-level reasoning and classification tasks.

Related papers

• Selective Ensemble Based on Preference-Directed Multi-Objective Bandits [90.75513823660775]
  We formalize a sequential decision problem under partially specified linear preferences.<n>We then propose a preference-directed upper confidence bound (PrefUCB) algorithm.<n> Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate our method.
  arXiv  Detail & Related papers  (2026-06-20T07:52:46Z)
• Data Bias Mitigation under Coverage Constraints & The Price of Fairness [1.94517837762427]
  We extend a recent bias mitigation framework to incorporate coverage constraints that enforce sufficient representation across groups.<n>Our solution trades small approximation errors in bias for greater data efficiency while satisfying coverage constraints.<n>This is essential both for legal compliance, where regulations may mandate specific fairness thresholds, and for data governance.
  arXiv  Detail & Related papers  (2026-06-18T16:43:34Z)
• A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability [0.20625936401496228]
  We study reliability techniques for large language models built on large language models (LLMs)<n>We show that each of these reliability techniques is a special case of one of six classical reliability operators.<n>We introduce a cost-nearest-neighbor router whose single Lagrangian knob traverses the quality-cost frontier.
  arXiv  Detail & Related papers  (2026-05-09T19:14:33Z)
• Real-Time AI Service Economy: A Framework for Agentic Computing Across the Continuum [10.35088918434398]
  This article shows that the structure of service-dependency graphs, modelled as DAGs, is a primary determinant of whether decentralised, price-based resource allocation can work reliably at scale.<n>We propose a hybrid management architecture in which cross-domain sub-graphs into resource slices that present a simpler, well-structured interface to the rest of the market.
  arXiv  Detail & Related papers  (2026-03-05T19:14:54Z)
• Phase Transition for Budgeted Multi-Agent Synergy [41.486076708302456]
  Multi-agent systems can improve reliability, yet under a fixed inference budget they often help, saturate, or even collapse.<n>We develop a minimal and calibratable theory that predicts these regimes from three binding constraints of modern agent stacks.
  arXiv  Detail & Related papers  (2026-01-24T05:32:50Z)
• Fair Bayesian Data Selection via Generalized Discrepancy Measures [11.013077130984973]
  We propose a data selection framework that ensures fairness by aligning group-specific posterior distributions of model parameters and sample weights with a shared central distribution.<n>Our framework supports flexible alignment via various distributional discrepancy measures, including Wasserstein distance, maximum mean discrepancy, and $f$-divergence.<n> Experiments on benchmark datasets show that our method consistently outperforms existing data selection and model-based fairness methods in both fairness and accuracy.
  arXiv  Detail & Related papers  (2025-11-10T12:28:04Z)
• Rebalancing with Calibrated Sub-classes (RCS): A Statistical Fusion-based Framework for Robust Imbalanced Classification across Modalities [16.993547305381327]
  Rebalancing with Calibrated Sub-classes (RCS) is a novel distribution calibration framework for robust imbalanced classification.<n>RCS fuses statistical information from the majority and intermediate class distributions via a weighted mixture of Gaussian components.
  arXiv  Detail & Related papers  (2025-10-10T00:06:13Z)
• Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning [71.30276778807068]
  We propose a unified framework that strategically coordinates sample pruning and token pruning.<n>Q-Tuning achieves a +38% average improvement over the full-data SFT baseline using only 12.5% of the original training data.
  arXiv  Detail & Related papers  (2025-09-28T13:27:38Z)
• Scalable Chain of Thoughts via Elastic Reasoning [61.75753924952059]
  Elastic Reasoning is a novel framework for scalable chain of thoughts.<n>It separates reasoning into two phases--thinking and solution--with independently allocated budgets.<n>Our approach produces more concise and efficient reasoning even in unconstrained settings.
  arXiv  Detail & Related papers  (2025-05-08T15:01:06Z)
• Powerful batch conformal prediction for classification [0.9374652839580183]
  A calibration sample of $n$ labeled examples is observed for inference on the label of an unlabeled example.<n>We provide a general recipe for valid inference with any combinations of conformal $p$-values.<n>We also introduce a more intensive method that aggregates batch scores and can be even more powerful.
  arXiv  Detail & Related papers  (2024-11-04T16:32:13Z)
• Minimax and Communication-Efficient Distributed Best Subset Selection with Oracle Property [0.358439716487063]
  The explosion of large-scale data has outstripped the processing capabilities of single-machine systems. Traditional approaches to distributed inference often struggle with achieving true sparsity in high-dimensional datasets. We propose a novel, two-stage, distributed best subset selection algorithm to address these issues.
  arXiv  Detail & Related papers  (2024-08-30T13:22:08Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Online covariance estimation for stochastic gradient descent under Markovian sampling [20.02012768403544]
  Convergence rates of order $Obig(sqrtd,n-1/8(log n)1/4big)$ are established under state-dependent and state-independent Markovian sampling. Our method is applied to the strategic classification with logistic regression, where adversaries adaptively modify features during training to affect target class classification.
  arXiv  Detail & Related papers  (2023-08-03T00:21:30Z)
• Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach [72.19525160912943]
  We first theoretically demonstrate the inherent connection between distribution shift, data perturbation, and model weight perturbation. We then analyze the sufficient conditions to guarantee fairness for the target dataset. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR)
  arXiv  Detail & Related papers  (2023-03-06T17:19:23Z)
• Robustifying Markowitz [3.154269505086154]
  The heavy-tail characteristics of financial time series are in fact the cause for these erratic fluctuations of weights. We present a toolbox for stabilizing costs and weights for global minimum Markowitz portfolios. We demonstrate that robustified portfolios reach the lowest turnover compared to shrinkage-based and constrained portfolios.
  arXiv  Detail & Related papers  (2022-12-28T18:09:14Z)
• Quantization for decentralized learning under subspace constraints [61.59416703323886]
  We consider decentralized optimization problems where agents have individual cost functions to minimize subject to subspace constraints. We propose and study an adaptive decentralized strategy where the agents employ differential randomized quantizers to compress their estimates. The analysis shows that, under some general conditions on the quantization noise, the strategy is stable both in terms of mean-square error and average bit rate.
  arXiv  Detail & Related papers  (2022-09-16T09:38:38Z)
• Stream-based Active Learning with Verification Latency in Non-stationary Environments [6.883906273999368]
  We investigate the influence of finite, time-variable, and unknown verification delay, in the presence of concept drift on AL approaches. We propose PRopagate, a latency independent utility estimator which predicts the requested, but not yet known, labels. We empirically show that the proposed method consistently outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2022-04-14T08:51:15Z)
• Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity [55.29408396918968]
  We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification. Our contributions include both consistency and robustness by establishing top-$k$ consistency of LDR losses for multi-class classification. We propose a new adaptive LDR loss that automatically adapts the individualized temperature parameter to the noise degree of class label of each instance.
  arXiv  Detail & Related papers  (2021-12-30T00:27:30Z)
• Decentralized Local Stochastic Extra-Gradient for Variational Inequalities [125.62877849447729]
  We consider distributed variational inequalities (VIs) on domains with the problem data that is heterogeneous (non-IID) and distributed across many devices. We make a very general assumption on the computational network that covers the settings of fully decentralized calculations. We theoretically analyze its convergence rate in the strongly-monotone, monotone, and non-monotone settings.
  arXiv  Detail & Related papers  (2021-06-15T17:45:51Z)
• SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression [68.66245730450915]
  We develop an improved method for debiasing predictions and estimating frequentist uncertainty for practical datasets. Our main contribution is SLOE, an estimator of the signal strength with convergence guarantees that reduces the computation time of estimation and inference by orders of magnitude.
  arXiv  Detail & Related papers  (2021-03-23T17:48:56Z)
• Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer [72.5190560787569]
  In computer vision, learning from long tailed datasets is a recurring theme, especially for natural image datasets. Our proposal posits a meta-distributional scenario, where the data generating mechanism is invariant across the label-conditional feature distributions. This allows us to leverage a causal data inflation procedure to enlarge the representation of minority classes.
  arXiv  Detail & Related papers  (2020-11-25T00:13:11Z)
• Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification [54.22421582955454]
  We provide the first result of the optimal minimax guarantees for the excess risk for adversarially robust classification. Results are stated in terms of the Adversarial Signal-to-Noise Ratio (AdvSNR), which generalizes a similar notion for standard linear classification to the adversarial setting.
  arXiv  Detail & Related papers  (2020-06-29T21:06:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.