Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

• URL: http://arxiv.org/abs/2607.27275v1
• Date: Wed, 29 Jul 2026 12:40:04 GMT
• Title: Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents
• Abstract summary: On $2$-bench, across two open-weight model families in dense and MoE variants, quantization indeed looks free on the standard metric.<n>Quantization amplifies the failure the model already exhibits at full precision by up to 2.5$times$ in volume.<n>A targeted error-repair prompt, run for five telecom models at every precision, removes the damage exactly and only where it lives.
• Score: 19.600181756177303
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless. We test this claim for multi-turn, tool-calling agents, where it now matters most. On $τ^2$-bench, across two open-weight model families in dense and MoE variants and two domains (eight cells, 456 episodes each, at 16-, 8-, and 4-bit weights), quantization indeed looks free on the standard metric. No cell shows a score change that survives multiple-comparison correction, and in the cell that carries the largest process damage, equivalence testing bounds the change within $\pm$7.5 points. The process tells a different story. Quantization amplifies the failure the model already exhibits at full precision (tool-name hallucination in telecom, with the same directional trend in retail entity errors) by up to 2.5$\times$ in volume (+17.6 points per task), while creating essentially no new failures. The failure set is the same at every precision (rank correlation $\geq$ 0.94, 0.18% novel events). The score stays flat because the benchmark's ten-error budget absorbs the extra failures. Shrinking the budget to two errors re-exposes a score gap of 17 points, and it does so only in the one cell where quantization added error volume, exactly as the masking account predicts. A targeted error-repair prompt, run for five telecom models at every precision, removes the damage exactly and only where it lives. Both diagnostics, the per-channel error rate and success under a shrinking budget, come from logs benchmarks already collect; we suggest reporting them alongside task reward.

Related papers

• Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents [22.30468215041597]
  A tool-using agent is given the same task in a different language, does it still take the same steps?<n>We make the action policy the measured object across 8 models, 6 parallel benchmarks and 41 languages.
  arXiv  Detail & Related papers  (2026-08-11T16:18:34Z)
• Zero2Skill: Bootstrapping Robot Skills through Autonomous Data Collection, Training, and Deployment [53.913175773174636]
  Existing pipelines reduce human effort via self-resetting, VLM verification, or language-guided correction.<n>We present Zero2Skill, a human-robot symbiotic agentic system in which corrections are retained and reused across rounds.
  arXiv  Detail & Related papers  (2026-07-15T17:16:24Z)
• Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate [0.0]
  Mixed-precision quantization must decide which parts of a model to keep at higher precision.<n>We show that the loss from quantizing a set of layers can be reconstructed from per-layer or pairwise sensitivities measured in isolation.<n>We propose the coverage model $f(S)=cbigl (1-prod_iin S (1-a_i)bigr)$, which reproduces the measured variance profile of $f$ to within a few percent from its $L$ fitted break-rates.
  arXiv  Detail & Related papers  (2026-07-14T02:08:33Z)
• PAEMS: Precise and Adaptive Error Model for Superconducting Quantum Processors [6.599826548609388]
  This paper introduces PAEMS: a precise and adaptive qubit error model.<n> Experiments on IBM's QPUs have demonstrated a 19.5$times$, 9.3$times$, and 5.2$times$ reduction in timelike, spacelike, and spacetime error correlation.<n>It also outperforms the accuracy of Google's SI$71000 error model by 58$sim3% on multiple quantum platforms.
  arXiv  Detail & Related papers  (2026-03-31T08:46:18Z)
• Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems [20.846301581161978]
  Failure attribution in multi-agent systems is a critical yet unsolved challenge.<n>Current methods treat this as a pattern recognition task over long conversation logs.<n>A2P Scaffolding transforms failure attribution from pattern recognition into a structured causal inference task.
  arXiv  Detail & Related papers  (2025-09-12T16:51:15Z)
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models [61.474101404805545]
  Diffusion models can generate high-quality images, but as they scale, rising memory demands and higher latency pose deployment challenges.<n>We propose SVDQuant, a new 4-bit quantization paradigm to overcome this limitation.<n>We reduce the memory usage for the 12B FLUX.1 models by 3.5$times$, achieving 3.0$times$ speedup over the 4-bit weight-only quantization (W4A16) baseline.
  arXiv  Detail & Related papers  (2024-11-07T18:59:58Z)
• Demonstrating a long-coherence dual-rail erasure qubit using tunable transmons [59.63080344946083]
  We show that a "dual-rail qubit" consisting of a pair of resonantly coupled transmons can form a highly coherent erasure qubit. We demonstrate mid-circuit detection of erasure errors while introducing $ 0.1%$ dephasing error per check. This work establishes transmon-based dual-rail qubits as an attractive building block for hardware-efficient quantum error correction.
  arXiv  Detail & Related papers  (2023-07-17T18:00:01Z)
• WR-ONE2SET: Towards Well-Calibrated Keyphrase Generation [57.11538133231843]
  Keyphrase generation aims to automatically generate short phrases summarizing an input document. The recently emerged ONE2SET paradigm generates keyphrases as a set and has achieved competitive performance. We propose WR-ONE2SET which extends ONE2SET with an adaptive instance-level cost Weighting strategy and a target Re-assignment mechanism.
  arXiv  Detail & Related papers  (2022-11-13T09:56:24Z)
• Suppressing quantum errors by scaling a surface code logical qubit [147.2624260358795]
  We report the measurement of logical qubit performance scaling across multiple code sizes. Our system of superconducting qubits has sufficient performance to overcome the additional errors from increasing qubit number. Results mark the first experimental demonstration where quantum error correction begins to improve performance with increasing qubit number.
  arXiv  Detail & Related papers  (2022-07-13T18:00:02Z)
• Exponential suppression of bit or phase flip errors with repetitive error correction [56.362599585843085]
  State-of-the-art quantum platforms typically have physical error rates near $10-3$. Quantum error correction (QEC) promises to bridge this divide by distributing quantum logical information across many physical qubits. We implement 1D repetition codes embedded in a 2D grid of superconducting qubits which demonstrate exponential suppression of bit or phase-flip errors.
  arXiv  Detail & Related papers  (2021-02-11T17:11:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.