Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair

• URL: http://arxiv.org/abs/2607.24604v1
• Date: Mon, 27 Jul 2026 16:05:23 GMT
• Title: Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair
• Abstract summary: We study the gap between finding a correct patch and retaining, verifying, and submitting it.<n>We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation.
• Score: 36.56438114281786
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generate--test--revise loops are common in coding agents, but repetition alone provides no reliability guarantee. We study the gap between finding a correct patch and retaining, verifying, and submitting it. A sealed five-seed study over 30 HumanEval repairs produces 900 three-revision trajectories. Under forced revision, current correctness with current traces falls from 0.820 after one revision to 0.673 after two, although ever-correct rises to 0.847. Two common-state studies use 2,430 branches from identical frozen programs to remove post-treatment risk-set bias. In a prespecified 14B replication, stale traces harm 34/135 correct starts versus 4/135 with current traces, a 22.2-point increase (task-cluster 95\% CI $[8.9,37.0]$, exact Holm $p=0.0337$). A prospective 540-rollout policy eliminates observed correct-start harm but reduces wrong-start repair and fails its joint criterion. Repository experiments over 24 bugs and four coder stacks expose floor effects and component heterogeneity without Holm-significant effects. We therefore separate admission, preservation, grounded certification, competence, and liveness. We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation that binds verifier evidence to exact code states, preserves verified checkpoints, and emits auditable admission receipts. The implementation is an executable specification and conformance artifact, not evidence of improved repair competence or calibrated verifier dependence.

Related papers

• ExecCritic: Learn to Test, Test to Improve for Coding Agents [68.42713285082912]
  Execution feedback can guide coding agents toward correct repository repairs, but only when the tests capture the behavior requested by the issue.<n>We introduce ExecCritic, combining a test--verify--revise scaffold with a role-specific reinforcement learning recipe for training agents within it.<n>The scaffold separates test construction from source-code repair: a Test agent independently generates repository-native tests, a fail-closed harness qualifies and freezes them, and a Repair agent revises source code from their execution feedback without changing the tests.
  arXiv  Detail & Related papers  (2026-09-08T17:53:37Z)
• Abstention Errors and Segment Support in CUAD: An Auditable Contract-Extraction Case Study [0.0]
  This paper presents a retrospective, reproducible evaluation of a fixed public RoBERTa checkpoint on CUAD's published 102-contract test split.<n>Of 30,464 returned candidate strings, 19,562 occur on questions with no annotated answer.<n>The system answers 1,335 of 2,938 such questions, a 45.4% false-positive rate.
  arXiv  Detail & Related papers  (2026-09-07T21:26:53Z)
• From Runnable to Verifiable: An Independent Reproducibility Study of LLM/Agent-Driven Vulnerability Validation Artifacts [4.093009863847175]
  Security research artifacts are increasingly produced by agent-driven vulnerabilities.<n>The gap between emphpublicly available, emphrunnable, emphsignal-producing, and emphsemantically confirmed artifacts is poorly measured.<n>We conduct a pre-registered consensus audit of this literature.
  arXiv  Detail & Related papers  (2026-08-10T13:05:02Z)
• Validation Evidence in LLM Repair Agents: How Much of What Passes Actually Tests the Bug? [1.6787220460106658]
  BSG-VA captures each validation command at its exact working-tree state, extracts a test-only patch, and replays the command on the original buggy code.<n>The captured outcome and the replay results assign every event an evidence role, from gold-aligned bug-discriminating through regression-only to misleading.<n>A three-arm experiment tests whether returning the B-replay outcome to the agent changes this pattern.
  arXiv  Detail & Related papers  (2026-07-30T22:32:14Z)
• DualityCert: Verifier-Gated Language-Model Repair of Broken Duality Claims in Quantum Field Theory [0.0]
  We present DualityCert, a symbolic verifier for candidate Seiberg-duality claims in four-dimensional N=1 quiver gauge theories.<n>A claim that passes receives a consistency certificate, which states that no tested inconsistency was found, not that the duality is proven.<n>We use the verifier as a repair environment for language-model agents, which receive a deliberately broken claim and must edit it until it certifies.
  arXiv  Detail & Related papers  (2026-07-26T11:39:49Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Proof-or-Stop: Don't Trust the Agent, Trust the Evidence -- Loop Engineering for Verifiable Evidence-Gated Lifecycle Control [5.796499740822509]
  We present Proof-or-Stop Lifecycle Control.<n>It permits lifecycle transitions only when fresh, tracked-source-state-bound, mechanically verifiable evidence satisfies the relevant gate.<n>We evaluate an open-source implementation through mechanism tests, a powered control-policy ablation, and operated self-application evidence.
  arXiv  Detail & Related papers  (2026-07-16T12:06:21Z)
• Structured Feedback Improves Repair in an LLM Agent Loop [2.8890052855500143]
  LLM agents often retry after external rejects a candidate, but the interface between validation and the next model call remains underspecified.<n>We introduce VeriHarness, a code-controlled agent loop in which models generate candidates while external validators control acceptance, budgets, and traces.<n>We use it to compare raw diagnostics with feedback that identifies the failure location, observed value, and admissible alternatives.
  arXiv  Detail & Related papers  (2026-07-15T06:14:45Z)
• SEVA: Self-Evolving Verification Agent with Process Reward for Fact Attribution [6.908637308550535]
  SEVA is a structured verification agent that emits evidence alignments, step-by-step reasoning chains, calibrated confidence, and a six-category error diagnosis.<n>On ClearFacts, SEVA-3B matches GPT-4o-mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output.
  arXiv  Detail & Related papers  (2026-06-29T02:37:13Z)
• Code Correctness Signals in LLM Hidden States: Pre-Generation Probing and Repair Geometry [0.0]
  The correctness of the model's first-attempt code is linearly decodable from the prompt-final hidden state.<n>On 236 cleaned cases where the model attempts to repair a failed first attempt, the hidden state shift from the failing attempt to its repair carries a statistically detectable contrastive direction.
  arXiv  Detail & Related papers  (2026-06-12T15:06:15Z)
• Converted, Not Equivalent: Benchmarking Codebase Conversion via Observational Equivalence [56.25095230687242]
  Coding agents often over-trust their own local validation routines and declare success on artifacts that satisfy surface checks.<n>This problem is especially acute in conversion, where prior evaluation is outcome-driven.<n>Across blind conversion attempts, the best system reaches 26.7--28.9% overall pass rate despite Spec pass rates up to 91.1%.<n>This suggests that failures stem more from contract-misaligned self-validation than from limited budget or backbone strength.
  arXiv  Detail & Related papers  (2026-05-27T19:57:15Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Affordance Agent Harness: Verification-Gated Skill Orchestration [45.231685718099264]
  Affordance grounding requires identifying where and how an agent should interact in open-world scenes.<n>We propose Affordance Agent Harness, a closed-loop runtime that unifies skills with an evidence store and cost control.
  arXiv  Detail & Related papers  (2026-05-01T13:45:16Z)
• Are Benchmark Tests Strong Enough? Mutation-Guided Diagnosis and Augmentation of Regression Suites [49.16055123488827]
  Insufficiently strong test suites can admit plausible yet semantically incorrect patches, inflating reported success rates.<n>We introduce STING, a framework for targeted test augmentation that uses semantically altered program variants as diagnostic stressors.<n>STING produces 1,014 validated tests spanning 211 instances and increases patch-region line and branch coverage by 10.8% and 9.5%.
  arXiv  Detail & Related papers  (2026-04-02T01:13:40Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.