Learning Adaptive LLM Decoding

• URL: http://arxiv.org/abs/2603.09065v2
• Date: Sat, 14 Mar 2026 00:54:46 GMT
• Title: Learning Adaptive LLM Decoding
• Abstract summary: We learn adaptive decoding policies that dynamically select sampling strategies at inference time, conditioned on available compute resources.<n>We introduce lightweight decoding adapters trained with reinforcement learning and verifiable terminal rewards.<n>Experiments show that the learned adapters improve the accuracy-budget tradeoff.
• Score: 6.643962667713069
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decoding from large language models (LLMs) typically relies on fixed sampling hyperparameters (e.g., temperature, top-p), despite substantial variation in task difficulty and uncertainty across prompts and individual decoding steps. We propose to learn adaptive decoding policies that dynamically select sampling strategies at inference time, conditioned on available compute resources. Rather than fine-tuning the language model itself, we introduce lightweight decoding adapters trained with reinforcement learning and verifiable terminal rewards (e.g. correctness on math and coding tasks). At the sequence level, we frame decoding as a contextual bandit problem: a policy selects a decoding strategy (e.g. greedy, top-k, min-p) for each prompt, conditioned on the prompt embedding and a parallel sampling budget. At the token level, we model decoding as a partially observable Markov decision process (POMDP), where a policy selects sampling actions at each token step based on internal model features and the remaining token budget. Experiments on the MATH and CodeContests benchmarks show that the learned adapters improve the accuracy-budget tradeoff: on MATH, the token-level adapter improves Pass@1 accuracy by up to 10.2% over the best static baseline under a fixed token budget, while the sequence-level adapter yields 2-3% gains under fixed parallel sampling. Ablation analyses support the contribution of both sequence- and token-level adaptation.

Related papers

• Compute Where it Counts: Self Optimizing Language Models [10.058821474955177]
  We study dynamic budget allocation for autoregressive decoding.<n>We train the policy with group-relative policy optimization on teacher-forced episodes.<n>Our reward trades off language-model quality against soft penalties that encourage episode-average budget usage.
  arXiv  Detail & Related papers  (2026-05-11T17:27:15Z)
• Adaptive Decoding via Test-Time Policy Learning for Self-Improving Generation [0.5199807441687141]
  We introduce a reinforcement learning-based sampler that treats decoding as sequential decision-making and learns a lightweight policy to adjust sampling parameters at test-time.<n>We evaluate summarization datasets including BookSum, arXiv, and WikiHow.
  arXiv  Detail & Related papers  (2026-03-19T02:44:42Z)
• $V_1$: Unifying Generation and Self-Verification for Parallel Reasoners [69.66089681814013]
  $V_$ is a framework that unifies generation and verification through efficient pairwise ranking.<n>$V_$-Infer improves Pass@1 by up to $10%$ over pointwise verification.<n>$V_$-PairRL achieves $7$--$9%$ test-time scaling gains over standard RL and pointwise joint training.
  arXiv  Detail & Related papers  (2026-03-04T17:22:16Z)
• Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding [21.609237262034636]
  Autoregressive decoding in large language models (LLMs) requires $mathcalO(n)$ sequential steps for $n$ tokens.<n>We propose Learning to Parallel Decode (Learn2PD), a framework that trains a lightweight and adaptive filter model to predict, for each token position, whether the current prediction matches the final output.<n>This learned filter approximates an oracle parallel decoding strategy that unmasks tokens only when correctly predicted.
  arXiv  Detail & Related papers  (2025-09-29T17:59:54Z)
• Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence [131.41894248194995]
  We propose context-oriented decomposition adaptation (CorDA), a novel method that initializes adapters in a task-aware manner.<n>Thanks to the task awareness, our method enables two optional adaptation modes, knowledge-preserved mode (KPM) and instruction-previewed mode (IPM)
  arXiv  Detail & Related papers  (2025-06-16T07:55:14Z)
• PATS: Process-Level Adaptive Thinking Mode Switching [53.53401063490537]
  Current large-language models (LLMs) typically adopt a fixed reasoning strategy, either simple or complex, for all questions, regardless of their difficulty.<n>This neglect of variation in task and reasoning process complexity leads to an imbalance between performance and efficiency.<n>Existing methods attempt to implement training-free fast-slow thinking system switching to handle problems of varying difficulty, but are limited by coarse-grained solution-level strategy adjustments.<n>We propose a novel reasoning paradigm: Process-Level Adaptive Thinking Mode Switching (PATS), which enables LLMs to dynamically adjust their reasoning strategy based on the difficulty of each step, optimizing the balance between
  arXiv  Detail & Related papers  (2025-05-25T17:58:50Z)
• CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning [101.81127587760831]
  Current fine-tuning methods build adapters widely of the context of downstream task to learn, or the context of important knowledge to maintain.<n>We propose CorDA, a Context-oriented Decomposition Adaptation method that builds learnable task-aware adapters.<n>Our method enables two options, the knowledge-preserved adaptation and the instruction-previewed adaptation.
  arXiv  Detail & Related papers  (2024-06-07T19:10:35Z)
• Token-Level Adaptation of LoRA Adapters for Downstream Task Generalization [0.0]
  This paper introduces a method for adapting LoRA adapters in smaller-sized language models to arbitrary downstream tasks. Our method employs a gradient-free routing function to choose a weighted combination of experts without increasing the compute requirements for training or inference.
  arXiv  Detail & Related papers  (2023-11-17T20:07:54Z)
• Actor-Critic based Improper Reinforcement Learning [61.430513757337486]
  We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process. We propose two algorithms: (1) a Policy Gradient-based approach; and (2) an algorithm that can switch between a simple Actor-Critic scheme and a Natural Actor-Critic scheme.
  arXiv  Detail & Related papers  (2022-07-19T05:55:02Z)
• Meta-learning via Language Model In-context Tuning [16.306733033119897]
  The goal of meta-learning is to learn to adapt to a new task with only a few labeled examples. We propose $textitin-context tuning, which recasts adaptation and prediction. We benchmark our method on two collections of text classification tasks: LAMA and BinaryClfs.
  arXiv  Detail & Related papers  (2021-10-15T02:29:09Z)
• Adaptive Task Sampling for Meta-Learning [79.61146834134459]
  Key idea of meta-learning for few-shot classification is to mimic the few-shot situations faced at test time. We propose an adaptive task sampling method to improve the generalization performance.
  arXiv  Detail & Related papers  (2020-07-17T03:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.