ADAPT: Learning Task Mixtures for Budget-Constrained Instruction Tuning

• URL: http://arxiv.org/abs/2512.04555v1
• Date: Thu, 04 Dec 2025 08:17:05 GMT
• Title: ADAPT: Learning Task Mixtures for Budget-Constrained Instruction Tuning
• Abstract summary: ADAPT is a meta-learning algorithm that emphlearns task sampling proportions under an explicit token budget for instruction tuning.<n>We conduct evaluations on 11 out-of-domain benchmarks spanning reasoning, reading comprehension, code generation, and instruction following.
• Score: 1.0039548765955955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose ADAPT, a meta-learning algorithm that \emph{learns} task sampling proportions under an explicit token budget for multi-task instruction tuning. Instead of fixing task weights by hand, \adapt{} maintains a continuous distribution over tasks and updates it via meta-gradients of a smooth worst-case validation objective, inducing an adaptive curriculum that allocates more tokens to useful tasks while avoiding collapse. We instantiate ADAPT on three $\sim$1B-parameter open-weight LLMs (Gemma-3-1B, LLaMA-3.2-1B, Qwen-0.6B), training on 20 Natural Instructions task types under budgets of $1\%$, $5\%$, and $10\%$ of the available supervised tokens, and compare against strong supervised fine-tuning baselines with uniform and size-proportional mixing. We conduct evaluations on 11 out-of-domain benchmarks spanning reasoning, reading comprehension, code generation, and instruction following, we find that ADAPT matches or slightly improves average downstream performance relative to the best static mixture, while using fewer effective training tokens and reallocating budget toward harder, benchmark-aligned tasks.

Related papers

• One Policy, Any Budget: Internalizing Budget-Aware Search via Reinforcement Learning [41.55260785497756]
  We propose AnySearch, a framework that enables a single policy to perform budget-aware search under any budget constraint.<n>In the first phase, we train the agent with explicit budget state injection and structured reasoning prompts.<n>In the second phase, the scaffold is removed and the agent learns to operate autonomously under adaptively sampled budget constraints.
  arXiv  Detail & Related papers  (2026-09-01T07:12:32Z)
• A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks [25.713629634281077]
  We propose TASTE: Task Synthesis from Tool Sequence Evolution, an automatic method that generates challenging tasks with broader tool-use coverage.<n> TASTE selects representative sequences from the pool via clustering, instantiates them into complete benchmark tasks, and refines them through difficulty evolution.<n>Our results suggest high scores on existing benchmarks often reflect saturation rather than robust task-solving ability.
  arXiv  Detail & Related papers  (2026-05-27T14:45:59Z)
• Learning Adaptive LLM Decoding [6.643962667713069]
  We learn adaptive decoding policies that dynamically select sampling strategies at inference time, conditioned on available compute resources.<n>We introduce lightweight decoding adapters trained with reinforcement learning and verifiable terminal rewards.<n>Experiments show that the learned adapters improve the accuracy-budget tradeoff.
  arXiv  Detail & Related papers  (2026-03-10T01:15:26Z)
• BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning [82.925106913459]
  Reinforcement finetuning (RFT) is a key technique for aligning Large Language Models (LLMs) with human preferences and enhancing reasoning.<n>We introduce BOTS, a unified framework for Bayesian Online Task Selection in RFT reinforcement finetuning.
  arXiv  Detail & Related papers  (2025-10-30T11:15:23Z)
• Robust-Multi-Task Gradient Boosting [6.920276126310231]
  Multi-task learning (MTL) has shown effectiveness in exploiting shared information across tasks to improve generalization.<n>We propose Robust-Multi-Task Gradient Boosting (R-MTGB), a novel boosting framework that explicitly models and adapts to task heterogeneity during training.<n>R-MTGB structures the learning process into three blocks: (1) learning shared patterns, (2) partitioning sequential tasks into outliers and non-outliers with regularized parameters, and (3) fine-tuning task-specific predictors.
  arXiv  Detail & Related papers  (2025-07-15T15:31:12Z)
• Aligning Instruction Tuning with Pre-training [61.50161961371844]
  We propose Aligning Instruction Tuning with Pre-training (AITP) to align instruction tuning with pre-training distributions.<n>We show consistent performance improvements with AITP on three fully open large language models (LLMs) across eight benchmarks.
  arXiv  Detail & Related papers  (2025-01-16T08:27:40Z)
• From Instance Training to Instruction Learning: Task Adapters Generation from Instructions [29.452006810725184]
  This paper focuses on simulating human learning to address the shortcomings of instance training.<n>We introduce Task Adapters Generation from Instructions (TAGI), which automatically constructs the task-specific model.<n>We evaluate TAGI on the Super-Natural Instructions and P3 datasets.
  arXiv  Detail & Related papers  (2024-06-18T08:14:28Z)
• Robust Multi-Task Learning with Excess Risks [24.695243608197835]
  Multi-task learning (MTL) considers learning a joint model for multiple tasks by optimizing a convex combination of all task losses. Existing methods use an adaptive weight updating scheme, where task weights are dynamically adjusted based on their respective losses to prioritize difficult tasks. We propose Multi-Task Learning with Excess Risks (ExcessMTL), an excess risk-based task balancing method that updates the task weights by their distances to convergence.
  arXiv  Detail & Related papers  (2024-02-03T03:46:14Z)
• Data-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning [85.66907881270785]
  We propose a data curriculum method, namely Data-CUBE, that arranges the orders of all the multi-task data for training. In the task level, we aim to find the optimal task order to minimize the total cross-task interference risk. In the instance level, we measure the difficulty of all instances per task, then divide them into the easy-to-difficult mini-batches for training.
  arXiv  Detail & Related papers  (2024-01-07T18:12:20Z)
• Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks [101.40633115037983]
  Instruction tuning (IT) achieves impressive zero-shot generalization results by training large language models (LLMs) on a massive amount of diverse tasks with instructions. How to select new tasks to improve the performance and generalizability of IT models remains an open question. We propose active instruction tuning based on prompt uncertainty, a novel framework to identify informative tasks, and then actively tune the models on the selected tasks.
  arXiv  Detail & Related papers  (2023-11-01T04:40:05Z)
• FAMO: Fast Adaptive Multitask Optimization [48.59232177073481]
  We introduce Fast Adaptive Multitask Optimization FAMO, a dynamic weighting method that decreases task losses in a balanced way. Our results indicate that FAMO achieves comparable or superior performance to state-of-the-art gradient manipulation techniques.
  arXiv  Detail & Related papers  (2023-06-06T15:39:54Z)
• ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning [59.08197876733052]
  Auxiliary-Task Learning (ATL) aims to improve the performance of the target task by leveraging the knowledge obtained from related tasks. Sometimes, learning multiple tasks simultaneously results in lower accuracy than learning only the target task, known as negative transfer. ForkMerge is a novel approach that periodically forks the model into multiple branches, automatically searches the varying task weights.
  arXiv  Detail & Related papers  (2023-01-30T02:27:02Z)
• Multi-Head Adapter Routing for Cross-Task Generalization [56.75667096355806]
  Polytropon learns an inventory of adapters and a routing function that selects a subset of adapters for each task during both pre-training and few-shot adaptation. We find that routing is most beneficial during multi-task pre-training rather than during few-shot adaptation.
  arXiv  Detail & Related papers  (2022-11-07T19:35:55Z)
• Boosting Natural Language Generation from Instructions with Meta-Learning [43.64522457686405]
  Recent work has shown that language models (LMs) trained with multi-task. textitinstructional learning (MTIL) can solve diverse NLP. tasks with improved performance compared to prompt tuning. In this paper we investigate whether meta-learning applied to MTIL can further improve generalization to unseen tasks in a zero-shot setting.
  arXiv  Detail & Related papers  (2022-10-20T22:23:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.