Gauge dependence and structured-output corruption in sign-branched repetition penalties: measurements across models, inference stacks, and alternative repetition controls

• URL: http://arxiv.org/abs/2607.09791v1
• Date: Thu, 09 Jul 2026 05:11:22 GMT
• Title: Gauge dependence and structured-output corruption in sign-branched repetition penalties: measurements across models, inference stacks, and alternative repetition controls
• Abstract summary: Penalty is not well-defined.<n>It corrupts structured output.<n>On 200 real-world schemas, theta=1.3 drops the rate of valid, schema-conformant output from 97% to 23%.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The multiplicative repetition penalty shipped across the LLM inference ecosystem (HuggingFace, vLLM, llama.cpp, and a dozen further engines) branches on the sign of each raw logit (divide positives by theta, multiply negatives). But the softmax is unchanged by adding a constant to every logit, so a model's logit zero-point is arbitrary, and the sign-branch reads that arbitrary point. The sign-branch is itself the accepted fix for an earlier bug, so the accepted fix branches on a quantity the training objective leaves unconstrained. Two measurable consequences follow. (1) The penalty is not well-defined: re-centring a model's logits by a constant is a provable no-op at theta=1, yet at a routine theta=1.3 it changes 58-96% of greedy tokens, where subtractive and normalized penalties change none; real checkpoints sit at widely different zero-points, so a fixed repetition_penalty is a different operation on every model. (2) It corrupts structured output: on 200 real-world JSON schemas, theta=1.3 drops the rate of valid, schema-conformant output from 97% to 23%. In our measurements, applying the penalty to normalized log-probabilities instead of raw logits removes both effects. HuggingFace already ships that operator (LogitNormalization); today it is off by default and applied after the penalty. This note gives the mechanism, the measurements (five models up to 7B, base and RLHF, on WikiText-103 prefixes; two code models on HumanEval and JSONSchemaBench; both effects replicated inside vLLM and llama.cpp through their own samplers on the same inputs), and the normalized variant.

Related papers

• Verification-Aware Training for Speculative Decoding [57.84976863792784]
  We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision.<n>VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure.<n>VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks.
  arXiv  Detail & Related papers  (2026-08-31T01:42:10Z)
• Nested Byte-Level Vocabularies Are Cheap to Deploy and Expensive to Share: A Pre-Registered Negative Result [0.0]
  We show how one language model can operate at several vocabulary sizes.<n>A control token indicates the active size, and be deployed at any trained size by slicing its embedding and output head.<n>A control with neither cap token nor output restriction is equally robust, attributing this benefit to multi-granularity training rather than conditioning.
  arXiv  Detail & Related papers  (2026-08-28T10:13:32Z)
• TraceCompiler: Skill-Guided Mining and Compilation of LLM Agent Traces into Mostly Deterministic Workflows [2.435006380732194]
  We present TraceCompiler, a skill-guided system that compiles noisy traces into trajectories.<n>Every hard edge carries an auditable evidence, and ambiguous relations are marked suspected.<n>We measure call reduction but not offline compilation cost, so we claim no net efficiency result.
  arXiv  Detail & Related papers  (2026-08-03T01:05:06Z)
• Abliteration Is Not a Scalpel: Off-Target Effects of Refusal Removal on Decision Disposition Across Model Families [51.56484100374058]
  Abliteration - deleting a model's refusal direction from its weights - is the standard recipe behind popular "uncensored" open-weight models.<n>As a disposition probe we use 21,600 decisions under uncertainty, replayed through a frozen pipeline so the decision-layer model is the only variable.<n>Three effects replicate across both families (weeks-clustered CIs excluding zero)<n>A fourth effect reverses sign: the same operation makes Gemma-abliterated less confident and Qwen-abliterated more (family CIs non-overlapping)
  arXiv  Detail & Related papers  (2026-07-19T22:27:00Z)
• Lossless but Not Free: An Empirical Anatomy of Speculative Decoding on Consumer Hardware [0.0]
  Single-stream autoregressive decoding of large language models is bound by memory bandwidth.<n>A small draft model proposes $K$ tokens autoregressively, the target model scores all of them in one batched pass.<n>We present a from-scratch, device-agnostic (CUDA/MPS/CPU) implementation and an empirical study across five draft/target backend configurations.
  arXiv  Detail & Related papers  (2026-07-19T15:01:35Z)
• Bayesian Repetition Penalty: A Principled Adjacent-Conditional Framework for Reversing Attention Collapse in Autoregressive Language Models [9.240704665480939]
  We present a principled framework that penalises or compensates anomalous confidence arising from collapsed generation patterns.<n>The resulting self-normalising penalty ratio $R=f(m,n,p)/f(np,n,p)$ requires no ad hoc standardisation and admits a closed-form logit offset with zero approximation error.
  arXiv  Detail & Related papers  (2026-07-17T17:01:57Z)
• Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models [0.0]
  We introduce a complementary diagnostic, operator-on-F, that compares a model's k-step latent pushforward to the environment's on an observable subset F.<n>On a TD-MPC2 size sweep over cheetah-run, reward-prediction error stays within [0.028, 0.091] for every model size.
  arXiv  Detail & Related papers  (2026-07-05T19:09:07Z)
• Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation [0.0]
  Per-token counterfactual credit estimation asks which token in a language-model rollout caused the final answer to be right or wrong.<n>We measure what that assumption costs on a stock inference engine, with a three-pass design.<n>Across six configurations and three models, re-feeding changes the credit estimate at rates 14-28 percentage points above the replica floor.
  arXiv  Detail & Related papers  (2026-06-14T06:09:16Z)
• Articulate but Wrong: Self-Review Failures in LLM-Based Code Modernization [1.0164694825170502]
  Large language model (LLM) agents are increasingly used to migrate legacy code to modern stacks.<n>We run 1,980 real modernization calls across 11 production LLMs from 7 distinct families on a balanced 60-snippet legacy-Python-2 corpus.<n>We find that semantic-preservation drift is prevalent and sharply separable from a cleanly-controlled baseline.
  arXiv  Detail & Related papers  (2026-05-20T05:00:31Z)
• Disposition Distillation at Small Scale: A Three-Arc Negative Result [0.0]
  Internal draft reported +33.9-point MCAS and +15.3-point HumanEval gains on a Qwen3-0.6B student; a second-pass sanity check falsified both numbers before publication.<n>We find no operator that moves judge-measured disposition without damaging content or collapsing into stylistic mimicry.<n>We contribute a three-arc negative result with mechanism, a two-failure-mode taxonomy for linear h_last probes, and an honest falsification pipeline that converts the class of false positives we ourselves produced into publishable negatives.
  arXiv  Detail & Related papers  (2026-04-13T17:40:31Z)
• CaTok: Taming Mean Flows for One-Dimensional Causal Image Tokenization [122.88484422855934]
  We present CaTok, a 1D causal image tokenizer with a MeanFlow decoder.<n>By selecting tokens over time intervals, CaTok learns causal 1D representations that support both fast one-step generation and high-fidelity multi-step sampling.<n> Experiments demonstrate that CaTok achieves state-of-the-art results on ImageNet reconstruction, reaching 0.75 FID, 22.53 PSNR and 0.674 SSIM.
  arXiv  Detail & Related papers  (2026-03-06T16:39:17Z)
• Universal Adversarial Suffixes Using Calibrated Gumbel-Softmax Relaxation [9.099589602551573]
  We study universal adversarial suffixes that, when appended to any input, broadly reduce accuracy across tasks and models.<n>Our approach learns the suffix in a differentiable "soft" form using Gumbel-Softmax relaxation and then discretizes it for inference.<n>A single suffix trained on one model transfers effectively to others, consistently lowering both accuracy and calibrated confidence.
  arXiv  Detail & Related papers  (2025-12-09T00:03:39Z)
• Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production [55.76222360698305]
  We explore a class of supervised training objectives that allow a language model to dynamically and autonomously scale the number of compute steps used for each input token.<n>For any token, the model can request additional compute steps by emitting a don't know> output.<n>We find that the CYB model requests additional steps when doing so improves accuracy, and the model adapts its processing time to token-level complexity and context.
  arXiv  Detail & Related papers  (2025-10-13T21:07:05Z)
• Pipeline Parallelism is All You Need for Optimized Early-Exit Based Self-Speculative Decoding [73.67253077506672]
  Large language models (LLMs) deliver impressive generation quality, but incur very high inference cost.<n>Early-exit based self-speculative decoding (EESD) has emerged to mitigate this cost.<n>We propose Pipeline-Parallel Self-Speculative Decoding (PPSD) that fully pipelines the draft and verification work.
  arXiv  Detail & Related papers  (2025-09-19T04:51:41Z)
• Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent [97.64313409741614]
  We propose to enforce a emphconsistency property which states that predictions of the model on its own generated data are consistent across time. We show that our novel training objective yields state-of-the-art results for conditional and unconditional generation in CIFAR-10 and baseline improvements in AFHQ and FFHQ.
  arXiv  Detail & Related papers  (2023-02-17T18:45:04Z)
• Improved techniques for deterministic l2 robustness [63.34032156196848]
  Training convolutional neural networks (CNNs) with a strict 1-Lipschitz constraint under the $l_2$ norm is useful for adversarial robustness, interpretable gradients and stable training. We introduce a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer. We significantly advance the state-of-the-art for standard and provable robust accuracies on CIFAR-10 and CIFAR-100.
  arXiv  Detail & Related papers  (2022-11-15T19:10:12Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.