Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks

• URL: http://arxiv.org/abs/2606.27474v1
• Date: Thu, 25 Jun 2026 18:52:24 GMT
• Title: Speculative Refinement: A Hybrid Autoregressive Diffusion Decoding Strategy and Its Behavior Across Benchmarks
• Abstract summary: We use Speculative Refinement (SpecRef) to warm-start a masked diffusion language model from an AR draft.<n>We evaluate SpecRef across six benchmarks (HumanEval, MBPP, GSM8K, BBH, ARC-Challenge, HellaSwag) with three distinct evaluation protocols.<n>These observations apply to any multi-stage or non-autoregressive generation pipeline and point toward more diagnostic evaluation practices.
• Score: 1.9800951131982487
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How should we evaluate generation systems that combine autoregressive (AR) and diffusion decoding? We study this question through Speculative Refinement (SpecRef), a training-free hybrid method that warm-starts a masked diffusion language model from an AR draft using entropy-guided selective masking. Evaluating SpecRef across six benchmarks (HumanEval, MBPP, GSM8K, BBH, ARC-Challenge, HellaSwag) with three distinct evaluation protocols (execution-based pass@1, exact-match, log-likelihood scoring), we surface several findings relevant beyond our specific system: (1) code benchmarks conflate structural discovery with logical correctness: providing a syntactic scaffold lifts accuracy from near zero to over 20% without changing the model, indicating that much of the baseline failure is structural; (2) a refinement tension phenomenon where multi-stage correction degrades already-correct tokens, exposing benchmark saturation ceilings invisible to single-model evaluation; (3) log-likelihood and generative evaluation produce different model rankings for the same model pair, suggesting they measure different capabilities; (4) standard Python post-processing silently breaks code evaluation for non-AR generators. These observations apply to any multi-stage or non-autoregressive generation pipeline and point toward more diagnostic evaluation practices.

Related papers

• Verification-Aware Training for Speculative Decoding [57.84976863792784]
  We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision.<n>VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure.<n>VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks.
  arXiv  Detail & Related papers  (2026-08-31T01:42:10Z)
• The RAT: A Unified Bayesian Model for RAG Evaluation [8.650407684623714]
  We introduce a Bayesian evaluation framework that jointly models retrieval success, abstention behavior, and answer correctness.<n>We apply the framework to 27 RAG configurations across three datasets, three retrievers, and three generators.<n>We extend the model to incorporate LLM-as-a-judge annotations as calibrated noisy observations.
  arXiv  Detail & Related papers  (2026-08-25T15:54:30Z)
• Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics [49.443264461057645]
  Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.<n>Progress in both paradigms is overwhelmingly tracked by generative perplexity (gen-PPL): the per-token negative log-likelihood of samples under a frozen autoregressive (AR) scorer.<n>We argue that this metric is unsound. By construction, gen-PPL measures only predictability under the scoring AR, not grammaticality or semantic coherence.
  arXiv  Detail & Related papers  (2026-06-07T02:35:56Z)
• ClaHF: A Human Feedback-inspired Reinforcement Learning Framework for Improving Classification Tasks [11.138973570127206]
  We propose ClaHF, a human feedback-inspired reinforcement learning framework for text classification.<n>ClaHF integrates preference modeling and RL optimization into the classification pipeline without requiring additional human annotations.<n>Results demonstrate that ClaHF consistently improves both classification performance and confidence calibration across diverse language models.
  arXiv  Detail & Related papers  (2026-05-17T14:00:01Z)
• Towards Generalized Image Manipulation Localization via Score-based Model [38.03898206912958]
  We propose DiffIML, a novel framework that introduces score-based generative modeling to IML.<n>Under this formulation, diffusion models serve as an effective numerical solver for the learned score function.<n>Experiments on eight non-generative and three generative benchmarks demonstrate that DiffIML consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-16T08:39:54Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• DVD: A Robust Method for Detecting Variant Contamination in Large Language Model Evaluation [24.086354908256293]
  textbfDVD is a single-sample detector that models the local output distribution induced by temperature sampling.<n>We construct the first benchmark for variant contamination across two domains Omni-MATH and SuperGPQA.<n>textbfDVD consistently outperforms perplexity-based, Min-$k$%++, edit-distance (CDD), and embedding-similarity baselines.
  arXiv  Detail & Related papers  (2026-01-08T12:48:40Z)
• Toward Reliable Machine Unlearning: Theory, Algorithms, and Evaluation [1.7767466724342065]
  We introduce Adrial Machine UNlearning (AMUN), which surpasses prior state-of-the-art methods for image classification based on SOTA MIA scores.<n>We show that existing methods fail in replicating a retrained model's behavior by introducing a nearest-neighbor membership inference attack (MIA-NN)<n>We then propose a fine-tuning objective that mitigates this leakage by approximating, for forget-class inputs, the distribution over remaining classes that a model retrained from scratch would produce.
  arXiv  Detail & Related papers  (2025-12-07T20:57:25Z)
• RHINO: Guided Reasoning for Mapping Network Logs to Adversarial Tactics and Techniques with Large Language Models [9.065322387043546]
  We introduce RHINO, a framework that decomposes Large Language Models into three interpretable phases mirroring human reasoning.<n> RHINO bridges the semantic gap between low-level observations and adversarial intent while improving output reliability through structured reasoning.<n>Our results demonstrate that RHINO significantly enhances the interpretability and scalability of threat analysis, offering a blueprint for deploying LLMs in operational security settings.
  arXiv  Detail & Related papers  (2025-10-16T02:25:46Z)
• DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation [68.19756761027351]
  Diffusion large language models (dLLMs) are compelling alternatives to autoregressive (AR) models.<n>We investigate their denoising processes and reinforcement learning methods.<n>Our work provides deeper insight into the machinery of dLLM generation and offers an effective, diffusion-native RL training framework.
  arXiv  Detail & Related papers  (2025-06-25T17:35:47Z)
• COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement [80.18490952057125]
  Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. We propose Context-Wise Order-Agnostic Language Modeling (COrAL) to overcome these challenges. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally.
  arXiv  Detail & Related papers  (2024-10-12T23:56:19Z)
• Unified Classification and Rejection: A One-versus-All Framework [47.58109235690227]
  We build a unified framework for building open set classifiers for both classification and OOD rejection. By decomposing the $ K $-class problem into $ K $ one-versus-all (OVA) binary classification tasks, we show that combining the scores of OVA classifiers can give $ (K+1) $-class posterior probabilities. Experiments on popular OSR and OOD detection datasets demonstrate that the proposed framework, using a single multi-class classifier, yields competitive performance.
  arXiv  Detail & Related papers  (2023-11-22T12:47:12Z)
• Visualizing Classifier Adjacency Relations: A Case Study in Speaker Verification and Voice Anti-Spoofing [72.4445825335561]
  We propose a simple method to derive 2D representation from detection scores produced by an arbitrary set of binary classifiers. Based upon rank correlations, our method facilitates a visual comparison of classifiers with arbitrary scores. While the approach is fully versatile and can be applied to any detection task, we demonstrate the method using scores produced by automatic speaker verification and voice anti-spoofing systems.
  arXiv  Detail & Related papers  (2021-06-11T13:03:33Z)
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
  We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We put forward an alternative measure of anomaly score to replace the reconstruction-based metric. Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T08:26:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.