AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs

• URL: http://arxiv.org/abs/2608.26004v1
• Date: Wed, 26 Aug 2026 16:50:02 GMT
• Title: AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs
• Abstract summary: AsymSpec is an asymmetric speculative decoding framework for agentic pipelines.<n>AsymSpec reaches $approx 90%$ of full-context accuracy on average, delivering $1.3$--$1.7times$ throughput speedups.<n>Results show that asymmetric context access yields substantial gains precisely when compression discards critical reasoning signals.
• Score: 17.842477184182556
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the accuracy-overhead trade-off. We propose AsymSpec, an asymmetric speculative decoding framework that breaks this symmetry: a lightweight drafter reads the full input while the large verifier operates on the compressed view. The drafter steers the verifier via a contrastive $δ$-fusion of logits, modulated by a divergence-aware acceptance gate that preserves verification stability and high draft acceptance rates. Evaluated across four agentic capabilities and two end-to-end agent benchmarks, AsymSpec reaches $\approx 90\%$ of full-context accuracy on average, delivering $1.3$--$1.7\times$ throughput speedups at $0.2$--$0.3\times$ the compute cost on isolated text capabilities. These results show that asymmetric context access yields substantial gains precisely when compression discards critical reasoning signals.

Related papers

• Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models [65.49950267695267]
  We introduce Omni2LoRA, a framework for efficient parametric memory compression via coherence-preserving context distillation.<n>By converting multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time multimodal-token load to zero.
  arXiv  Detail & Related papers  (2026-08-10T07:49:10Z)
• A Cost-Aware, Paired Protocol for Auditing Dynamic Tool Synthesis in Agentic Video Question Answering [2.1096366377985185]
  Agentic Video Questioning (VideoQA) systems invoke tools during inference, but their tool libraries are fixed.<n>We propose a cost-aware paired protocol for auditing tool-augmented video agents.
  arXiv  Detail & Related papers  (2026-07-01T21:01:57Z)
• Raw-Curve Quantum Fingerprints: A Mahalanobis Authentication Framework with Drift Early Warning and Adversarial Detection [11.819995530903567]
  Quantum cloud platforms are poised to deliver powerful computing capabilities, but users have no direct means to verify which physical device executes their workload.<n>This lack of transparency enables hardware substitution attacks, where a malicious adversary could redirect a job to a substituted or inferior processor.<n>We present a general authentication framework that addresses this problem by constructing multi-dimensional quantum fingerprints from raw measurement data.
  arXiv  Detail & Related papers  (2026-06-10T04:10:21Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference [56.297697169678095]
  WISV (Wireless-Informed Semantic Verification) is a novel distributed speculative decoding framework.<n>We show that WISV achieves up to a 60.8% increase in accepted length, a 37.3% reduction in interaction rounds, and a 31.4% improvement in end-to-end latency.<n>We validate WISV on a hardware testbed comprising an NVIDIA Jetson AGX Orin and an A40-equipped server.
  arXiv  Detail & Related papers  (2026-04-20T01:29:56Z)
• RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding [80.12789199134511]
  Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency.<n>We propose $textbfRACER, a lightweight and training-free method that integrates retrieved exact patterns with logit-driven future cues.<n>Experiments on Spec-Bench, HumanEval, and MGSM-ZH demonstrate that RACER consistently accelerates inference, achieving more than $2times$ speedup over autoregressive decoding.
  arXiv  Detail & Related papers  (2026-04-16T11:23:55Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference [11.957170239588535]
  Speculative decoding accelerates inference by using a draft model to look ahead.<n>Prior methods partially reduce draft cost but either degrade acceptance or introduce overheads that limit scaling.<n>We present Mirror Speculative Decoding (Mirror-SD), an inference algorithm that breaks the latency-acceptance tradeoff.
  arXiv  Detail & Related papers  (2025-10-15T05:22:57Z)
• SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning [14.020244011380063]
  SpecReason is a system that accelerates LRM inference.<n>It exploits the semantic flexibility of thinking tokens in preserving final-answer accuracy.<n>It achieves $1.4-3.0times$ speedup over vanilla LRM inference.
  arXiv  Detail & Related papers  (2025-04-10T16:05:19Z)
• DASA: Delay-Adaptive Multi-Agent Stochastic Approximation [64.32538247395627]
  We consider a setting in which $N$ agents aim to speedup a common Approximation problem by acting in parallel and communicating with a central server. To mitigate the effect of delays and stragglers, we propose textttDASA, a Delay-Adaptive algorithm for multi-agent Approximation.
  arXiv  Detail & Related papers  (2024-03-25T22:49:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.