ContraFix: Agentic Vulnerability Repair via Differential Runtime Evidence and Skill Reuse

• URL: http://arxiv.org/abs/2605.17450v1
• Date: Sun, 17 May 2026 13:48:25 GMT
• Title: ContraFix: Agentic Vulnerability Repair via Differential Runtime Evidence and Skill Reuse
• Abstract summary: Large language model (LLM) agents are increasingly used for automated vulnerability repair.<n>Recent empirical results show that these agents still struggle with real-world vulnerabilities.<n>We present ContraFix, an agentic framework that couples runtime evidence with reusable repair skills.
• Score: 10.503895811137095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents are increasingly used for automated vulnerability repair (AVR), where repository-level reasoning enables them to inspect context and produce source-code patches. However, recent empirical results show that these agents still struggle with real-world vulnerabilities. Their main failure mode is semantic misunderstanding: choosing a repair direction that does not match the root cause. We identify two reasons for this gap. Existing agents usually reason from the failing execution alone. A crash report can pinpoint where the program failed, but it does not reveal which variable or state transition, among many candidates near the fault site, separates the crashing behavior from safe execution. As a result, agents often produce symptom-oriented patches instead of causal fixes. Moreover, evidence collected for one vulnerability is rarely retained, so similar cases in later repositories must be diagnosed again from scratch. We present ContraFix, an agentic AVR framework that couples differential runtime evidence with reusable repair skills. Its Mutator constructs PoC variants that straddle the failure boundary; its Analyzer inserts state probes around the fault region and summarizes divergences between crashing and non-crashing executions into a repair specification; and its Patcher converts the specification into verified source patches. Each successful repair updates a two-track skill base containing repair specifications and mutation strategies, which are retrieved through a three-tier policy for future instances. On SEC-Bench (C/C++, 200 instances) and PatchEval (Go, Python, JavaScript, 225 instances), ContraFix with GPT-5-mini resolves 84.0% and 73.8% of the tasks, respectively, achieving state-of-the-art performance on both benchmarks while costing less than one-third of the strongest comparable baseline.

Related papers

• PatchBench: Evaluating AI Agents for Vulnerability Patching [9.083611812336539]
  We propose a new benchmark for evaluating AI agents on realistic vulnerability patching tasks.<n> PatchBench selects vulnerabilities whose ground-truth fixes lie outside the crash stack and uses vulnerability transplant and code mutations to migrate historical vulnerabilities into new repository contexts.<n>Across 11 state-of-the-art agents, including the top three AIxCC agents, the original PoC-only validations the patching task solve rate of agents by 1.83$times$ on average.
  arXiv  Detail & Related papers  (2026-09-03T16:44:22Z)
• CodeMechanic: Bug-Property-Guided Program Mitigation [23.976869967674435]
  We present CodeMechanic, a bug-property-guided system for generating constrained mit- igations for memory corruption.<n>Instead of asking an LLM to generate a permanent repair, CodeMechanic reconstructs the violated memory-safety property from the crash.<n>CodeMechanic produces 3.4x - 4.3x more patches semantically equivalent to developer-written repairs.
  arXiv  Detail & Related papers  (2026-08-23T08:06:47Z)
• CausalRepair: Bridging the Causality Gap in Large Language Model-Based Automated Program Repair via Dual-Slicing [18.917995731082073]
  CausalRepair is a conversation-driven APR framework based on minimal causal context.<n>It correctly fixes 313 bugs on Defects4J, outperforming state-of-the-art approaches such as ReinFix and TSAPR.
  arXiv  Detail & Related papers  (2026-08-11T07:56:32Z)
• Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework [49.16055123488827]
  coding agents powered by large language models can now generate code patches directly from bug reports.<n>No mechanism independently verifies whether it truly resolves the reported problem.<n>We propose RETRACE, a training-free post-generation verification framework.
  arXiv  Detail & Related papers  (2026-08-09T22:59:13Z)
• Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixes [55.114939648705395]
  Large language models (LLMs) have made automated program repair increasingly practical for real-world bugs.<n>Bug reproduction tests (BRTs) help close this gap by turning a bug report into an executable, bug-specific signal.<n>We propose CoHarden, a co-generation framework that uses the Lax signal as an in-loop convergence criterion.
  arXiv  Detail & Related papers  (2026-07-22T07:30:07Z)
• AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents [83.37969790806069]
  LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.<n>We present Agent DebuggerX, an open-source framework that organizes debug as a closed loop of Detect, Attribute, Recover, and Rerun.<n>At its core, Deep Debugger performs multi-turn root-cause diagnosis through global trajectory understanding.
  arXiv  Detail & Related papers  (2026-07-21T06:21:13Z)
• MemRepair: Hierarchical Memory for Agentic Repository-Level Vulnerability Repair [10.709112246848058]
  Modern software ecosystems face a rapidly growing number of disclosed vulnerabilities.<n>Most existing systems treat vulnerability repair as a single generation step over the currently visible code context.<n>We present MemRepair, a memoryaugmented agentic framework that formulates vulnerability repair as an iterative, experience-driven process.
  arXiv  Detail & Related papers  (2026-05-17T13:29:46Z)
• Patch2Vuln: Agentic Reconstruction of Vulnerabilities from Linux Distribution Binary Patches [3.93181912653522]
  This paper asks whether a language-model agent, restricted to local binary-derived evidence, can reconstruct the security meaning of Linux distribution updates.<n>Patch2Vuln is a local, resumable pipeline that extracts old/new ELF pairs, diffs them with Ghidra and Ghidriff, ranks changed functions, builds candidate dossiers, and asks an offline agent to produce a preliminary audit.<n>We evaluate Patch2Vuln on 25 Ubuntu.deb package pairs: 20 security-update pairs and five negative controls, all manually adjudicated against private source-patch and binary-function ground
  arXiv  Detail & Related papers  (2026-05-07T17:22:22Z)
• Mitigating Implicit Inconsistencies in Patch Porting [13.153399281876451]
  Implicit inconsistencies pose challenges for developers to resolve due to their non-local nature.<n>We propose MIP, which enables collaboration among an LLM, a compiler, and code analysis utilities.<n> Experiments on two representative scenarios, cross-fork and cross-branch patch porting, show that MIP successfully resolves more than twice as many patches as the best-performing baseline in both settings.
  arXiv  Detail & Related papers  (2026-04-02T06:33:31Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Test-Driven AI Agent Definition (TDAD): Compiling Tool-Using Agents from Behavioral Specifications [0.0]
  Test-Driven AI Agent Definition (TDAD) is a methodology that treats agent prompts as compiled artifacts.<n>We evaluate TDAD on SpecSuite-Core, a benchmark of four deeply-specified agents spanning policy compliance, grounded analytics, runbook adherence, and deterministic enforcement.
  arXiv  Detail & Related papers  (2026-03-09T18:04:54Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Where LLM Agents Fail and How They can Learn From Failures [62.196870049524364]
  Large Language Model (LLM) agents have shown promise in solving complex, multi-step tasks.<n>They amplify vulnerability to cascading failures, where a single root-cause error propagates through subsequent decisions.<n>Current systems lack a framework that can comprehensively understand agent error in a modular and systemic way.<n>We introduce the AgentErrorTaxonomy, a modular classification of failure modes spanning memory, reflection, planning, action, and system-level operations.
  arXiv  Detail & Related papers  (2025-09-29T18:20:27Z)
• What Do They Fix? LLM-Aided Categorization of Security Patches for Critical Memory Bugs [46.325755802511026]
  We developLM, a dual-method pipeline that integrates two approaches based on a Large Language Model (LLM) and a fine-tuned small language model.<n>LM successfully identified 111 of 5,140 recent Linux kernel patches addressing OOB or UAF vulnerabilities, with 90 true positives confirmed by manual verification.
  arXiv  Detail & Related papers  (2025-09-26T18:06:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.