Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols

• URL: http://arxiv.org/abs/2604.18245v2
• Date: Sun, 26 Apr 2026 15:08:07 GMT
• Title: Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols
• Abstract summary: We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks.<n>For each instance, the interface records a baseline correctness bit and a post-step correctness bit.<n>These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines.
• Score: 51.56484100374058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly deployed as protocols: structured multi-call procedures that spend additional computation to transform a baseline answer into a final one. These protocols are evaluated only by end-to-end accuracy, giving limited insight into when they help, when they hurt, and whether their behavior transfers under distribution shift or composition. We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks. For each instance, the interface records a baseline correctness bit $E_0\in\{0,1\}$ and a post-step correctness bit $E_1\in\{0,1\}$, separating correction ($E_0=0\to E_1=1$) from corruption ($E_0=1\to E_1=0$) through two rates: $c=\Pr(E_1=1\mid E_0=0)$ and $γ=\Pr(E_1=0\mid E_0=1)$. These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines. We identify three failure mechanisms. Under mixture shift, pooled estimates of $(c,γ)$ become biased when calibration and deployment mixtures differ; conditioning on a difficulty proxy restores stability without additional model calls. Under presentation contamination, selection protocols alter the interface through stable presentation artifacts when candidate content is fixed. Under state insufficiency, the correctness bit may not carry enough history for multi-step pipelines to compose predictably; a Markov factorization test identifies when composition is valid and where additional state is needed. When a protocol step passes these diagnostics, it becomes an auditable module: gated by estimated gain, conditioned on a difficulty proxy to correct mixture bias, and composed into multi-step pipelines with predictable accuracy. We demonstrate these ideas on synthetic mathematical tasks and on GSM8K, where the calibrated interface correctly predicts when protocol steps should be activated or suppressed.

Related papers

• Probabilistic Model Checking of Autoregressive Neural Sequence Models [4.570003973862485]
  Test-set accuracy is silent on two issues that matter when deploying autoregressive neural sequence models.<n>How much probability mass the system under test places on constraint-violating alternatives that are reachable under sampling.<n>We answer both with probabilistic model checking.
  arXiv  Detail & Related papers  (2026-09-01T07:37:55Z)
• Sound and Efficient Certification of High-Quality Qubit Operations: Theory and Experiment [23.48673361417131]
  We introduce a sound black-box certification protocol for a single-qubit gate based on a small set of fixed, deterministic sequences.<n>From the data, the protocol derives finite-sample bounds on the gate's rotation eigenvalue, a gauge-invariant property.<n>We prove that these spectral constraints imply, up to a physically meaningful unitary change of basis, a rigorous average gate-fidelity lower bound for every time-independent qubit model.
  arXiv  Detail & Related papers  (2026-08-17T18:03:00Z)
• Lottery Tickets Are Not Deployment Tickets [4.230349220081444]
  We study whether an accuracy-matched lottery ticket or another sparse challenger can replace an incumbent dense model without reconfiguring downstream decision logic.<n>Across experiments, sparse candidates repeatedly recover dense-reference accuracy yet remain behaviorally different.<n>In small-gap settings with fixed-threshold policy diagnostics, lottery-ticket replacement changes 7% to 10% of accept-review decisions.
  arXiv  Detail & Related papers  (2026-07-29T15:25:19Z)
• Semantic Drift in Bug Resolution: How Behavioral Signals Propagate from Reports to Tests and Patches [18.727291516223115]
  Desc2Fix is a framework for measuring semantic alignment between bug reports, triggering tests, and developer-written fixes.<n>We analyze 2,857 report-test-patch triplets from Defects4J and SWT-Bench using GPT-4o and DeepSeek-Chat.
  arXiv  Detail & Related papers  (2026-07-20T22:32:23Z)
• A Cost-Aware, Paired Protocol for Auditing Dynamic Tool Synthesis in Agentic Video Question Answering [2.1096366377985185]
  Agentic Video Questioning (VideoQA) systems invoke tools during inference, but their tool libraries are fixed.<n>We propose a cost-aware paired protocol for auditing tool-augmented video agents.
  arXiv  Detail & Related papers  (2026-07-01T21:01:57Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• Tight Convergence Rates for Online Distributed Linear Estimation with Adversarial Measurements [66.94250413799232]
  We study mean estimation of a random vector $X$ in a distributed parameter-server-worker setup.<n>Key challenges are adversarial measurements and asynchrony.<n>Results provide a unified finite-time characterization of robustness, identifiability, and statistical efficiency in distributed linear estimation with adversarial workers.
  arXiv  Detail & Related papers  (2026-04-07T11:45:55Z)
• Autonomous Hamiltonian certification and changepoint detection [1.2021700665093262]
  Environmental noise can cause calibrated Hamiltonian parameters to drift over time, necessitating expensive recalibration.<n>We develop efficient Hamiltonian certification and changepoint detection protocols in the autonomous setting.<n>Our approach enables quantum devices to autonomously monitor their own calibration status without requiring entangling systems, verification operations, or a trusted reference device.
  arXiv  Detail & Related papers  (2026-03-27T17:54:45Z)
• The Forecast After the Forecast: A Post-Processing Shift in Time Series [17.131164796761446]
  We propose a lightweight, architecture-agnostic way to boost deployed time series forecasters without retraining.<n>$$-Adapter learns tiny, bounded modules at two interfaces.<n>It can act as a feature selector by learning a sparse, horizon-aware mask over inputs to select important features.<n>It can also be used as a distribution calibrator to measure uncertainty.
  arXiv  Detail & Related papers  (2026-01-28T05:55:04Z)
• Phase-space entropy at acquisition reflects downstream learnability [54.4100065023873]
  We propose an acquisition-level scalar $S_mathcal B$ based on instrument-resolved phase space.<n>We show theoretically that (S_mathcal B) correctly identifies the phase-space coherence of periodic sampling.<n>$|S_mathcal B|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty emphwithout training.
  arXiv  Detail & Related papers  (2025-12-22T10:03:51Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Coverage-Guaranteed Speech Emotion Recognition via Calibrated Uncertainty-Adaptive Prediction Sets [0.0]
  Road rage, often triggered by emotional suppression and sudden outbursts, significantly threatens road safety by causing collisions and aggressive behavior.<n>Speech emotion recognition technologies can mitigate this risk by identifying negative emotions early and issuing timely alerts.<n>We propose a novel risk-controlled prediction framework providing statistically rigorous guarantees on prediction accuracy.
  arXiv  Detail & Related papers  (2025-03-24T12:26:28Z)
• On Computationally Efficient Multi-Class Calibration [9.032290717007065]
  Project calibration gives strong guarantees for all downstream decision makers. It ensures that the probabilities predicted by summing the probabilities assigned to labels in $T$ are close to some perfectly calibrated binary predictor.
  arXiv  Detail & Related papers  (2024-02-12T17:25:23Z)
• Semi-DETR: Semi-Supervised Object Detection with Detection Transformers [105.45018934087076]
  We analyze the DETR-based framework on semi-supervised object detection (SSOD) We present Semi-DETR, the first transformer-based end-to-end semi-supervised object detector. Our method outperforms all state-of-the-art methods by clear margins.
  arXiv  Detail & Related papers  (2023-07-16T16:32:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.