ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding

• URL: http://arxiv.org/abs/2604.14612v1
• Date: Thu, 16 Apr 2026 04:39:30 GMT
• Title: ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding
• Abstract summary: Self-speculative decoding is an inference technique for large language models designed to speed up generation without sacrificing output quality.<n>We propose ConfLayers, a dynamic plug-and-play approach to forming the draft model in self-speculative decoding via confidence-based intermediate layer skipping.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-speculative decoding is an inference technique for large language models designed to speed up generation without sacrificing output quality. It combines fast, approximate decoding using a compact version of the model as a draft model with selective re-evaluation by the full target model. Some existing methods form the draft model by dynamically learning which layers to skip during inference, effectively creating a smaller subnetwork to speed up computation. However, using heuristic-based approaches to select layers to skip can often be simpler and more effective. In this paper, we propose ConfLayers, a dynamic plug-and-play approach to forming the draft model in self-speculative decoding via confidence-based intermediate layer skipping. The process iteratively computes confidence scores for all layers, selects layers to skip based on an adaptive threshold, evaluates the performance of the resulting set, and updates the best selection until no further improvement is achieved or a maximum number of iterations is reached. This framework avoids the overhead and complexity of training a layer skipping policy and can provide more consistent speed-quality trade-offs while preserving the adaptivity of the draft model to diverse tasks and datasets. The performance evaluation of ConfLayers across different models and datasets shows that our novel approach offers up to 1.4x speedup over vanilla LLM generation.

Related papers

• SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration [13.454534256560558]
  Speculative decoding has emerged as a promising approach to accelerate autoregressive inference in large language models.<n>We propose a novel self-draft framework that suppresses spurious confidence via layer-wise temperature annealing in early-exit decision.<n>We show that our method achieves up to 2.33x wall-time speedup over standard autoregressive decoding.
  arXiv  Detail & Related papers  (2026-04-14T03:47:04Z)
• Fast Inference via Hierarchical Speculative Decoding [65.40448210801763]
  We introduce Hierarchical Speculative Decoding (HSD), an algorithm that stacks draft models into a hierarchy, where each model proposes tokens, and the next larger model verifies them in a single forward pass.<n>HSD gives up to 1.2x speed-up over the best single-draft baseline.
  arXiv  Detail & Related papers  (2025-10-22T15:56:19Z)
• Input Conditioned Layer Dropping in Speech Foundation Models [11.05223262950967]
  layer dropping ($mathcalLD$) skips fraction of the layers of a backbone network during inference to reduce the computational load.<n>We propose input-driven $mathcalLD$ that employs the network's input features and a lightweight layer selecting network to determine optimum combination of processing layers.
  arXiv  Detail & Related papers  (2025-07-10T17:39:03Z)
• CLaSp: In-Context Layer Skip for Self-Speculative Decoding [20.800300833576035]
  We propose CLaSp, an in-context layer-skipping strategy for self-speculative decoding.<n>Unlike prior methods, CLaSp does not require additional drafting modules or extra training.<n>CLaSp achieves a speedup of 1.3x 1.7x on LLaMA3 series models without altering the original distribution of the generated text.
  arXiv  Detail & Related papers  (2025-05-30T04:15:06Z)
• LESA: Learnable LLM Layer Scaling-Up [57.0510934286449]
  Training Large Language Models (LLMs) from scratch requires immense computational resources, making it prohibitively expensive.<n>Model scaling-up offers a promising solution by leveraging the parameters of smaller models to create larger ones.<n>We propose textbfLESA, a novel learnable method for depth scaling-up.
  arXiv  Detail & Related papers  (2025-02-19T14:58:48Z)
• COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement [80.18490952057125]
  Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. We propose Context-Wise Order-Agnostic Language Modeling (COrAL) to overcome these challenges. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally.
  arXiv  Detail & Related papers  (2024-10-12T23:56:19Z)
• Accelerating Inference in Large Language Models with a Unified Layer Skipping Strategy [67.45518210171024]
  Dynamic computation methods have shown notable acceleration for Large Language Models (LLMs) by skipping several layers of computations. We propose a Unified Layer Skipping strategy, which selects the number of layers to skip computation based solely on the target speedup ratio. Experimental results on two common tasks, i.e., machine translation and text summarization, indicate that given a target speedup ratio, the Unified Layer Skipping strategy significantly enhances both the inference performance and the actual model throughput.
  arXiv  Detail & Related papers  (2024-04-10T12:12:07Z)
• DynaLay: An Introspective Approach to Dynamic Layer Selection for Deep Networks [0.0]
  We introduce textbfDynaLay, an alternative architecture that features a decision-making agent to adaptively select the most suitable layers for processing each input. DynaLay reevaluates more complex inputs during inference, adjusting the computational effort to optimize both performance and efficiency. Our experiments demonstrate that DynaLay achieves accuracy comparable to conventional deep models while significantly reducing computational demands.
  arXiv  Detail & Related papers  (2023-12-20T05:55:05Z)
• Layer-wise Linear Mode Connectivity [52.6945036534469]
  Averaging neural network parameters is an intuitive method for the knowledge of two independent models. It is most prominently used in federated learning. We analyse the performance of the models that result from averaging single, or groups.
  arXiv  Detail & Related papers  (2023-07-13T09:39:10Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• Improving Reliability of Fine-tuning with Block-wise Optimisation [6.83082949264991]
  Finetuning can be used to tackle domain-specific tasks by transferring knowledge. We propose a novel block-wise optimization mechanism, which adapts the weights of a group of layers of a pre-trained model. The proposed approaches are tested on an often-used dataset, Tf_flower.
  arXiv  Detail & Related papers  (2023-01-15T16:20:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.