ConFu: Contemplate the Future for Better Speculative Sampling

• URL: http://arxiv.org/abs/2603.08899v1
• Date: Mon, 09 Mar 2026 20:11:06 GMT
• Title: ConFu: Contemplate the Future for Better Speculative Sampling
• Abstract summary: textbfConFu (Contemplate the Future) is a novel speculative decoding framework that enables draft models to anticipate the future direction of generation.<n>Our work is the first to bridge speculative decoding with continuous reasoning tokens, offering a new direction for accelerating LLM inference.
• Score: 40.48053935426729
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speculative decoding has emerged as a powerful approach to accelerate large language model (LLM) inference by employing lightweight draft models to propose candidate tokens that are subsequently verified by the target model. The effectiveness of this paradigm critically depends on the quality of the draft model. While recent advances such as the EAGLE series achieve state-of-the-art speedup, existing draft models remain limited by error accumulation: they condition only on the current prefix, causing their predictions to drift from the target model over steps. In this work, we propose \textbf{ConFu} (Contemplate the Future), a novel speculative decoding framework that enables draft models to anticipate the future direction of generation. ConFu introduces (i) contemplate tokens and soft prompts that allow the draft model to leverage future-oriented signals from the target model at negligible cost, (ii) a dynamic contemplate token mechanism with MoE to enable context-aware future prediction, and (iii) a training framework with anchor token sampling and future prediction replication that learns robust future prediction. Experiments demonstrate that ConFu improves token acceptance rates and generation speed over EAGLE-3 by 8--11% across various downstream tasks with Llama-3 3B and 8B models. We believe our work is the first to bridge speculative decoding with continuous reasoning tokens, offering a new direction for accelerating LLM inference.

Related papers

• Draft-OPD: On-Policy Distillation for Speculative Draft Models [49.23782868133977]
  Speculative decoding accelerates large language model inference by pairing a target model with a lightweight draft model whose proposed tokens are verified in parallel.<n>We propose Draft-OPD, which uses target-assisted rollout for stable continuations and replays drafting from the verification-exposed error positions.
  arXiv  Detail & Related papers  (2026-05-28T04:30:22Z)
• Training-free Dropout Sampling for Semantic Token Acceptance in Speculative Decoding [13.249778063956917]
  Speculative decoding accelerates large language model inference by proposing tokens with a lightweight draft model.<n>This work introduces DropMatch, a novel approach that matches draft tokens to the predictive distribution of the target model.<n> Experiments across multiple benchmarks demonstrate that our approach increases acceptance length while maintaining competitive task performance.
  arXiv  Detail & Related papers  (2026-02-11T04:53:33Z)
• PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations [52.67948063133533]
  Generative Recommendation has emerged as a promising paradigm, reformulating recommendation as a sequence-to-sequence generation task over hierarchical Semantic IDs.<n>Existing methods suffer from a critical issue we term Semantic Drift, where errors in early, high-level tokens irreversibly divert the generation trajectory into irrelevant semantic subspaces.<n>We propose Promise, a novel framework that integrates dense, step-by-step verification into generative models.
  arXiv  Detail & Related papers  (2026-01-08T07:38:46Z)
• Context-level Language Modeling by Learning Predictive Context Embeddings [79.00607069677393]
  We introduce textbfContextLM, a framework that augments standard pretraining with an inherent textbfnext-context prediction objective.<n>This mechanism trains the model to learn predictive representations of multi-token contexts, leveraging error signals derived from future token chunks.<n>Experiments on the GPT2 and Pythia model families, scaled up to $1.5$B parameters, show that ContextLM delivers consistent improvements in both perplexity and downstream task performance.
  arXiv  Detail & Related papers  (2025-10-23T07:09:45Z)
• Next Interest Flow: A Generative Pre-training Paradigm for Recommender Systems by Modeling All-domain Movelines [8.895768051554162]
  We propose a novel generative pre-training paradigm for e-commerce recommender systems.<n>Our model learns to predict the Next Interest Flow, a dense vector sequence representing a user's future intent.<n>We present the All-domain Moveline Evolution Network (AMEN), a unified framework implementing our entire pipeline.
  arXiv  Detail & Related papers  (2025-10-13T12:13:17Z)
• DONUT: A Decoder-Only Model for Trajectory Prediction [12.89335607622991]
  We propose DONUT, a Decoder-Only Network for Unrolling Trajectories.<n>We encode historical trajectories and predict future trajectories with a single autoregressive model.<n>We achieve new state-of-the-art results on the Argoverse 2 single-agent motion forecasting benchmark.
  arXiv  Detail & Related papers  (2025-06-07T16:24:29Z)
• PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation [4.031603850949324]
  We introduce a novel speculative decoding method that enables low-cost adaptation of autoregressive draft models into parallel draft models.<n>Our proposed conditional drop token method can improve draft model training efficiency by 3x.<n>On our optimized inference framework, PARD accelerates LLaMA3.1-8B inference by 4.08x, achieving 311.5 tokens per second.
  arXiv  Detail & Related papers  (2025-04-23T12:27:43Z)
• Latent Thought Models with Variational Bayes Inference-Time Computation [52.63299874322121]
  Latent Thought Models (LTMs) incorporate explicit latent thought vectors that follow an explicit prior model in latent space.<n>LTMs demonstrate superior sample and parameter efficiency compared to autoregressive models and discrete diffusion models.
  arXiv  Detail & Related papers  (2025-02-03T17:50:34Z)
• Graph-Structured Speculative Decoding [52.94367724136063]
  Speculative decoding has emerged as a promising technique to accelerate the inference of Large Language Models. We introduce an innovative approach utilizing a directed acyclic graph (DAG) to manage the drafted hypotheses. We observe a remarkable speedup of 1.73$times$ to 1.96$times$, significantly surpassing standard speculative decoding.
  arXiv  Detail & Related papers  (2024-07-23T06:21:24Z)
• Accelerating Production LLMs with Combined Token/Embedding Speculators [4.649953910785797]
  This report describes the design and training of novel speculative decoding draft models. By conditioning draft predictions on both context vectors and sampled tokens, we can train our speculators to efficiently predict high-quality n-grams.
  arXiv  Detail & Related papers  (2024-04-29T21:59:07Z)
• Decoding Speculative Decoding [4.56754610152086]
  Speculative Decoding is a technique to speed up inference for Large Language Models without sacrificing quality.<n>We study over 350 experiments with LLaMA-65B and OPT-66B using speculative decoding.<n>Our newly designed draft model can provide 111% higher throughput than existing draft models.
  arXiv  Detail & Related papers  (2024-02-02T16:15:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.