Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing

• URL: http://arxiv.org/abs/2604.18170v1
• Date: Mon, 20 Apr 2026 12:29:53 GMT
• Title: Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing
• Abstract summary: LLMs edit text and code by autoregressively regenerating the full output, even when most tokens appear verbatim in the input.<n>We study Copy-as-Decode, a decoding-layer mechanism that recasts edit generation as structured decoding over a two-primitive grammar.
• Score: 2.6382975801439836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs edit text and code by autoregressively regenerating the full output, even when most tokens appear verbatim in the input. We study Copy-as-Decode, a decoding-layer mechanism that recasts edit generation as structured decoding over a two-primitive grammar: <copy lines="i-j"/> references an input line range, <gen>...</gen> emits new content. A token-level FSM guarantees syntactic validity, and a serving-layer primitive updates the KV cache for each copy span via a single parallel-prefill forward rather than $N$ autoregressive steps -- sharing the parallel-forward kernel of speculative decoding but with input tokens as the draft and program-enforced acceptance replacing probabilistic verification. We report an upper-bound analysis that requires no end-to-end training. (i) Kernel speedup: on Qwen2.5-{1.5B, 7B}, copying $N$ tokens via parallel prefill is $6.8\times$--$303\times$ faster than autoregressive ($N \in [8, 512]$, A100 80GB bf16). (ii) Copy ceiling: on ProbeEdit and HumanEvalPack-Fix (Py/JS), $74$--$98\%$ of gold tokens are reachable under the line-level primitive; composed with the empirical kernel over each corpus's span histogram this yields a closed-form wall-clock bound of $29.0\times / 3.4\times / 4.2\times$ ($13.0\times$ pooled). A token-level extension reaches $91$--$99\%$ coverage with $4.5\times$--$6.5\times$ floors. (iii) Pipeline losslessness: oracle programs round-trip through the deterministic resolver on all $482$ cases, localizing any downstream failure to span selection rather than the mechanism. A perturbation study shows pooled EM drops from $100\%$ to $15.48\%$ under off-by-one noise. A fine-tuning pilot on Qwen2.5-Coder-1.5B lifts HEvalFix-Py EM from $0/33$ (untrained) to $12$--$17\%$, a learnability signal, not a production selector. Batched-serving integration and multi-file coverage are scoped as follow-up.

Related papers

• Fixed-order postselected CHSH reference acquisition for parity-constrained spatial-mode qubits on a commercial cloud photonic processor [0.0]
  We report a fixed-order Clauser-Horne-Shimony-Holt (CHSH) acquisition and reporting protocol for two encoded photonic qubits on Quandela's commercial, cloud-accessible Belenos processor.<n>Each logical qubit is a two-dimensional spatial-mode subspace in the seven-dimensional zero-sum sector of an eight-mode single-photon register.
  arXiv  Detail & Related papers  (2026-08-13T01:08:02Z)
• ForgettingOT: Certified Speculative Batching from Sinkhorn's Projective Forgetting [0.0]
  We show that the eigenmode of the fixed-point Jacobian $J_tstar=QP=Pstar P$ controls the strict correction tail.<n>ForgettingOT turns this nonlinear Perron--Frobenius fact into a certified executor for streams of related Sinkhorn problems.
  arXiv  Detail & Related papers  (2026-07-27T17:59:36Z)
• Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes [70.61868608402723]
  We introduce $textbfSelf-Correcting Coupled Markov Jump Processes (SC-CMJP)<n>In conjunction with SC-CMJP, we introduce $textttCO_texttt2textttJump$, a novel training-free single-pass sampler for joint multimodal geneneration.<n>For training and evaluation purposes, we have created and will release three large-scale joint multimodal generation corpora.
  arXiv  Detail & Related papers  (2026-07-14T18:39:29Z)
• ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation [72.54604107217669]
  textscpass@$k$ nearly vanishes after compression, yet textscpass@$k$ recovers substantially under repeated sampling.<n>Recovery should train on the compressed model's own on-policy states with dense token-level supervision.<n>We propose textbfshortopd, a short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes.
  arXiv  Detail & Related papers  (2026-07-14T17:50:50Z)
• Isolating LLM Alignment from Regex: Zero Coverage and Metric-Dependent Divergence Under Adversarial Mutation [51.56484100374058]
  Previous work found no measurable coverage gain from adding a live Gemini backend behind an active filter.<n>We introduce $L_5$-no-regex -- identical to $L_4$-real (Gemini-2.5-flash, token-budget cap, rate limit, output scrub) but with the nine-pattern filter disabled.<n>We evaluate it against adversarial probes across three sub-languages.
  arXiv  Detail & Related papers  (2026-06-12T18:54:24Z)
• The Security Budget of Code-LLM Prompt Hardening: Provable Limits Under Pass-Only Acceptance [0.0]
  We operationalize the floor as the emphTri-Audit Protocol, a two-axis reporting protocol that separates a prompt-side deductive registry attribute from a model-side empirical corollary.<n>A constrained best-of-family search over deterministic and guarded learned filters on CodeLlama-7B, Qwen2.5-Coder-7B/1.5B and DeepSeek-Coder-6.7B at $n=164$ yields the emphCross-Model Tri-Audit Invariance: of twenty-eight pass-preserving rows, twelve-changed-of-record learned-can
  arXiv  Detail & Related papers  (2026-06-02T08:22:14Z)
• DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts [32.57334288518084]
  Modern RL post-training methods such as GRPO and DAPO train on $N$ response sequences of $R$ tokens sampled from a shared prompt.<n>Standard FlashAttention replicates all $P$ prompt tokens $N times across both forward and backward passes.<n>We propose textDualAttention, the first Flash kernel variant eliminates shared-prompt replication during training.
  arXiv  Detail & Related papers  (2026-05-14T21:11:32Z)
• SURGE: SuperBatch Unified Resource-efficient GPU Encoding for Heterogeneous Partitioned Data [3.1624024957575982]
  SURGE is a streaming encoding system deployed in production to generate embeddings for over 800 million texts across 40,000 logical partitions.<n>On 10M texts with 4 NVIDIA L4768, SURGE delivers 26,413 texts/s -- matching fixed-times throughput using 12.6times less memory.
  arXiv  Detail & Related papers  (2026-05-01T19:51:50Z)
• RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding [80.12789199134511]
  Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency.<n>We propose $textbfRACER, a lightweight and training-free method that integrates retrieved exact patterns with logit-driven future cues.<n>Experiments on Spec-Bench, HumanEval, and MGSM-ZH demonstrate that RACER consistently accelerates inference, achieving more than $2times$ speedup over autoregressive decoding.
  arXiv  Detail & Related papers  (2026-04-16T11:23:55Z)
• Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding [21.609237262034636]
  Autoregressive decoding in large language models (LLMs) requires $mathcalO(n)$ sequential steps for $n$ tokens.<n>We propose Learning to Parallel Decode (Learn2PD), a framework that trains a lightweight and adaptive filter model to predict, for each token position, whether the current prediction matches the final output.<n>This learned filter approximates an oracle parallel decoding strategy that unmasks tokens only when correctly predicted.
  arXiv  Detail & Related papers  (2025-09-29T17:59:54Z)
• Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass [72.07642648108849]
  Superposed Decoding is a new decoding algorithm that generates $k$ drafts at the cost of one autoregressive inference pass. Superposed Decoding can be combined with other decoding strategies, resulting in universal coverage gains when scaling inference time compute.
  arXiv  Detail & Related papers  (2024-05-28T17:40:48Z)
• SpecTr: Fast Speculative Decoding via Optimal Transport [30.18181671899423]
  We develop a new autoregressive sampling algorithm called $textitSpecTr$, which provides speedup in decoding while ensuring that there is no quality degradation in the decoded output. We experimentally demonstrate that for state-of-the-art large language models, the proposed approach achieves a wall clock speedup of 2.13X, a further 1.37X speedup over speculative decoding on standard benchmarks.
  arXiv  Detail & Related papers  (2023-10-23T17:47:34Z)
• Deep Learning Meets Projective Clustering [66.726500395069]
  A common approach for compressing NLP networks is to encode the embedding layer as a matrix $AinmathbbRntimes d$. Inspired by emphprojective clustering from computational geometry, we suggest replacing this subspace by a set of $k$ subspaces.
  arXiv  Detail & Related papers  (2020-10-08T22:47:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.