Automatic Configuration of LLM Post-Training Pipelines

• URL: http://arxiv.org/abs/2603.18773v1
• Date: Thu, 19 Mar 2026 11:26:56 GMT
• Title: Automatic Configuration of LLM Post-Training Pipelines
• Abstract summary: AutoPipe is a budget-aware framework for configuration selection in LLM post-training.<n> offline, AutoPipe learns a dataset-conditioned learning-to-rank surrogate from historical runs.<n>Online, AutoPipe uses the offline guidance to steer Bayesian optimization and models dataset-specific deviations.
• Score: 4.833783769369
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM post-training pipelines that combine supervised fine-tuning and reinforcement learning are difficult to configure under realistic compute budgets: the configuration space is high-dimensional and heterogeneous, stages are strongly coupled, and each end-to-end evaluation is expensive. We propose AutoPipe, a budget-aware two-stage framework for configuration selection in LLM post-training. Offline, AutoPipe learns a dataset-conditioned learning-to-rank surrogate from historical runs, capturing within-dataset preferences and providing transferable guidance toward promising regions of the configuration space. Online, for a new dataset, AutoPipe uses the offline guidance to steer Bayesian optimization and models dataset-specific deviations with a Gaussian-process residual surrogate. To reduce evaluation cost, each trial is early-stopped and scored by a learned predictor that maps early training signals to a low-cost proxy for final post-training performance. Experiments on biomedical reasoning tasks show that AutoPipe consistently outperforms offline-only baselines and achieves comparable performance with the strongest online HPO baselines while using less than 10\% of their computational cost.

Related papers

• AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism [14.600392553338681]
  Large-language-models (LLMs) demonstrate enormous utility in long-context tasks.<n>Existing LLM training libraries do not provide easy to use abstractions to optimize for long-context training.<n>We introduce AutoSP: the first automated solution to automatically optimize LLM training for longer-contexts.
  arXiv  Detail & Related papers  (2026-04-29T18:31:29Z)
• AutoOR: Scalably Post-training LLMs to Autoformalize Operations Research Problems [54.593031581486116]
  We present AutoOR, a scalable synthetic data generation and reinforcement learning pipeline that trains LLMs to autoformalize optimization problems specified in natural language.<n>AutoOR generates verified training data from standard optimization forms and uses solver execution feedback as the reward signal for RL post-training.<n>We believe that methods such as AutoOR can significantly accelerate industrial decision-making with AI.
  arXiv  Detail & Related papers  (2026-04-18T03:24:54Z)
• Learning to Staff: Offline Reinforcement Learning and Fine-Tuned LLMs for Warehouse Staffing Optimization [15.738083856839749]
  We train custom Transformer-based policies using offline reinforcement learning on detailed historical state representations.<n>We explore LLMs operating on abstracted, human-readable state descriptions.<n>Our findings demonstrate that both approaches offer viable paths toward AI-assisted operational decision-making.
  arXiv  Detail & Related papers  (2026-03-25T23:56:25Z)
• The Chicken and Egg Dilemma: Co-optimizing Data and Model Configurations for LLMs [86.27977008139435]
  JoBS is an approach that uses a scaling-law-inspired performance predictor to aid Bayesian optimization.<n>We study JoBS's average regret and devise the optimal budget allocation to minimize regret.
  arXiv  Detail & Related papers  (2026-02-09T07:33:40Z)
• TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Training [6.7228358095570995]
  TwIST is a distributed training framework for efficient large language model sparsification.<n>It trains multipleworks in parallel, periodically aggregates their parameters, and resamples newworks during training.<n>It identifies high-qualityworks ("golden tickets") without requiring post-training procedures such as calibration or Hessian-based recovery.
  arXiv  Detail & Related papers  (2025-11-06T02:13:24Z)
• Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward [54.708851958671794]
  We propose a Data-Efficient Policy Optimization pipeline that combines optimized strategies for both offline and online data selection.<n>In offline phase, we curate a high-quality subset of training samples based on diversity, influence, and appropriate difficulty.<n>During online RLVR training, we introduce a sample-level explorability metric to dynamically filter samples with low exploration potential.
  arXiv  Detail & Related papers  (2025-09-01T10:04:20Z)
• Towards a Proactive Autoscaling Framework for Data Stream Processing at the Edge using GRU and Transfer Learning [0.0]
  We show how a GRU neural network forecasts the upstream load using real-world and synthetic DSP datasets.<n>A transfer learning framework integrates the predictive model into an online stream processing system.<n>The lightweight GRU model for load predictions recorded up to 1.3% SMAPE value on a real-world data set.
  arXiv  Detail & Related papers  (2025-07-19T12:47:50Z)
• Constrain Alignment with Sparse Autoencoders [45.131670081186]
  Feature-level constrained Preference Optimization is a novel method designed to simplify the alignment process while ensuring stability.<n>Our approach enjoys efficiency by using sparse features activated in a well-trained sparse autoencoder and the quality of sequential KL divergence.
  arXiv  Detail & Related papers  (2024-11-12T07:54:13Z)
• Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment [104.18002641195442]
  We introduce Self-Augmented Preference Optimization (SAPO), an effective and scalable training paradigm that does not require existing paired data. Building on the self-play concept, which autonomously generates negative responses, we further incorporate an off-policy learning pipeline to enhance data exploration and exploitation.
  arXiv  Detail & Related papers  (2024-05-31T14:21:04Z)
• Unleashing the Power of Pre-trained Language Models for Offline Reinforcement Learning [50.9692060692705]
  This paper introduces $textbfLanguage Models for $textbfMo$tion Control ($textbfLaMo$), a general framework based on Decision Transformers for offline RL.<n>Our framework highlights four crucial components:.<n>Initializing Decision Transformers with sequentially pre-trained LMs, (2) employing the LoRA fine-tuning method,.<n>In particular, our method demonstrates superior performance in scenarios with limited data samples.
  arXiv  Detail & Related papers  (2023-10-31T16:24:17Z)
• Tracking Performance of Online Stochastic Learners [57.14673504239551]
  Online algorithms are popular in large-scale learning settings due to their ability to compute updates on the fly, without the need to store and process data in large batches. When a constant step-size is used, these algorithms also have the ability to adapt to drifts in problem parameters, such as data or model properties, and track the optimal solution with reasonable accuracy. We establish a link between steady-state performance derived under stationarity assumptions and the tracking performance of online learners under random walk models.
  arXiv  Detail & Related papers  (2020-04-04T14:16:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.