EvidenT: An Evidence-Preserving Framework for Iterative System-Level Package Repair

• URL: http://arxiv.org/abs/2605.08621v1
• Date: Sat, 09 May 2026 02:29:57 GMT
• Title: EvidenT: An Evidence-Preserving Framework for Iterative System-Level Package Repair
• Abstract summary: We propose EvidenT, an evidence-preserving repair framework that decouples evidence management from tool execution.<n>We evaluate EvidenT on 219 real-world RISC-V package build failures.<n>Preliminary experiments achieve success rates of 41.77% on aarch64 and 46.99% on x86_64, demonstrating robustness across diverse hardware ecosystems.
• Score: 26.445772656438752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Frequent toolchain updates and growing ISA diversity have made system-level software package repair increasingly important. Diagnosing and repairing build failures remains challenging because failures involve heterogeneous evidence, dependency constraints, and architecture-specific build conventions. While recent LLM-based repair methods show promise for project-level source fixes, they struggle with system-level repair, where failures span multi-language artifacts such as build recipes, scripts, and source archives, and require iterative validation through external build services. In this paper, we first conduct a systematic empirical study of real-world system-level build failures. We find that 72% of failures stem from dependency and environment misconfigurations rather than isolated code defects, suggesting that effective repair must prioritize packaging logic and iterative feedback. Motivated by these insights, we propose EvidenT, an evidence-preserving repair framework that decouples iteration-aware evidence management from tool execution. EvidenT includes: (1) an external Build Service for reproducible execution and feedback; (2) an Evidence-Preserving Repair Controller that fuses repair history, knowledge context, and build artifacts; and (3) an automated Repair Orchestrator that invokes modular tools for failure localization and system-level repair in a closed-loop validation environment. We evaluate EvidenT on 219 real-world RISC-V package build failures. EvidenT repairs 118 packages (53.88%), outperforming state-of-the-art agentic baselines (20.55%) and direct LLM-based repair (1.83%). To assess architectural generality, we extend EvidenT to legacy ISAs by updating only ISA-specific knowledge context. Preliminary experiments achieve success rates of 41.77% on aarch64 and 46.99% on x86_64, demonstrating robustness across diverse hardware ecosystems.

Related papers

• SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents [63.65682461568621]
  We introduce SWE-Gate, a repository-level benchmark for software engineering agents that explicitly evaluates review constraint compliance alongside functional correctness.<n>SWE-Gate derives review constraints from real pull request review comments and synthesizes repository-level repair instances around these constraints.<n>Each instance provides separate functional and constraint tests, together with non-compliant and gold patches, enabling explicit separation between issue resolution capability and review constraint compliance.
  arXiv  Detail & Related papers  (2026-09-03T17:53:34Z)
• DepRepair: LLM-Based Source-Code Repair for Dependency Breaking Changes [26.444574129479463]
  DepRepair is a single-call LLM approach that grounds repair in structured upstream evidence through three components.<n>It attains the highest executable pass rate on each backbone, achieving 89.5% with GPT-5.5 and 82.1% with Claude Opus 4.6.
  arXiv  Detail & Related papers  (2026-07-20T13:58:13Z)
• Knowledge-Enhanced Agentic Vulnerability Repair [20.715227299482947]
  KeaRepair is a novel agentic approach that grounds patch generation in verified program facts and high-level vulnerability knowledge.<n>It significantly outperforms existing approaches on 55 C/C++ vulnerabilities.<n>When paired with Gemini-3.1-Pro, KeaRepair successfully repairs 46 vulnerabilities, achieving a repair rate of 83.64%.
  arXiv  Detail & Related papers  (2026-07-01T11:42:25Z)
• RAVEN: Agentic RAG for Automated Vulnerability Repair [48.8954115363348]
  We propose RAVEN, a scalable, efficient and autonomous framework for automated vulnerability repair.<n>RAVEN integrates an agentic retrieval-augmented generation (RAG) pipeline with controlled iterative repair in a unified framework.<n>RAVEN achieves an overall repair success rate of 83.13%, outperforming all existing state-of-the-art repair frameworks.
  arXiv  Detail & Related papers  (2026-06-21T19:18:06Z)
• MemRepair: Hierarchical Memory for Agentic Repository-Level Vulnerability Repair [10.709112246848058]
  Modern software ecosystems face a rapidly growing number of disclosed vulnerabilities.<n>Most existing systems treat vulnerability repair as a single generation step over the currently visible code context.<n>We present MemRepair, a memoryaugmented agentic framework that formulates vulnerability repair as an iterative, experience-driven process.
  arXiv  Detail & Related papers  (2026-05-17T13:29:46Z)
• ComBench: A Repo-level Real-world Benchmark for Compilation Error Repair [36.10273400046946]
  ComBench is the first repository-level, reproducible real-world benchmark for C/C++ compilation error repair.<n> ComBench is constructed through a novel, automated framework that mines real-world failures from GitHub CI histories.<n>Our experiments reveal a significant gap between a model's ability to achieve syntactic correctness and its ability to ensure semantic correctness.
  arXiv  Detail & Related papers  (2026-03-28T16:35:34Z)
• Detect--Repair--Verify for LLM-Generated Code: A Multi-Language, Multi-Granularity Empirical Study [10.18490328199727]
  Large language models can generate runnable software artifacts, but their security remains difficult to evaluate end to end.<n>This study examines that problem through a Detect--Repair--Verify (DRV) workflow, in which vulnerabilities are detected, repaired, and then rechecked with security and functional tests.<n>It addresses four gaps in current evidence: the lack of test-grounded benchmarks for LLM-generated artifacts, limited evidence on pipeline-level effectiveness, unclear reliability of detection reports as repair guidance, and uncertain repair trustworthiness under verification.
  arXiv  Detail & Related papers  (2026-03-24T18:18:30Z)
• FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair [45.42550492783294]
  Multimodal Automated Program Repair (MAPR) extends traditional program repair by requiring models to jointly reason over source code, textual issue descriptions, and visual artifacts such as GUI screenshots.<n>We propose FailureMem, a multimodal repair framework that integrates three key mechanisms: a hybrid workflow-agent architecture that balances structured localization with flexible reasoning, active perception tools that enable region-level visual grounding, and a Failure Memory Bank that converts past repair attempts into reusable guidance.
  arXiv  Detail & Related papers  (2026-03-18T15:24:22Z)
• Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition [53.50448142467294]
  RAIM is a multi-design and architecture-aware framework for repository-level feature addition.<n>It shifts away from linear patching by generating multiple diverse implementation designs.<n>Experiments on the NoCode-bench Verified dataset demonstrate that RAIM establishes a new state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-02T12:50:40Z)
• A Benchmark for Language Models in Real-World System Building [56.549267258789904]
  Cross-ISA software package repair is a critical task for ensuring the reliability of software deployment and the stability of modern operating systems.<n>We introduce a new benchmark designed for software package build repair across diverse architectures and languages.<n>We evaluate six state-of-the-art LLMs on the benchmark, and the results show that cross-ISA software package repair remains difficult and requires further advances.
  arXiv  Detail & Related papers  (2026-01-19T10:30:46Z)
• Can Language Models Go Beyond Coding? Assessing the Capability of Language Models to Build Real-World Systems [44.748487030119]
  Large language models (LLMs) have shown growing potential in software engineering.<n>Few benchmarks evaluate their ability to repair software during migration across instruction set architectures (ISAs)
  arXiv  Detail & Related papers  (2025-11-02T03:23:07Z)
• Where LLM Agents Fail and How They can Learn From Failures [62.196870049524364]
  Large Language Model (LLM) agents have shown promise in solving complex, multi-step tasks.<n>They amplify vulnerability to cascading failures, where a single root-cause error propagates through subsequent decisions.<n>Current systems lack a framework that can comprehensively understand agent error in a modular and systemic way.<n>We introduce the AgentErrorTaxonomy, a modular classification of failure modes spanning memory, reflection, planning, action, and system-level operations.
  arXiv  Detail & Related papers  (2025-09-29T18:20:27Z)
• Bridging Solidity Evolution Gaps: An LLM-Enhanced Approach for Smart Contract Compilation Error Resolution [2.967464333639626]
  Solidity, the dominant smart contract language, has rapidly evolved with frequent version updates to enhance security, functionality, and developer experience.<n>We conduct an empirical study to investigate the challenges in the Solidity version evolution and reveal that 81.68% of examined contracts encounter errors when compiled across different versions, with 86.92% of compilation errors.<n>We introduce SMCFIXER, a novel framework that integrates expert knowledge retrieval with LLM-based repair mechanisms for Solidity compilation error resolution.
  arXiv  Detail & Related papers  (2025-08-14T10:42:26Z)
• Specification-Guided Repair of Arithmetic Errors in Dafny Programs using LLMs [79.74676890436174]
  We present an APR tool for Dafny that uses formal specifications as oracles for fault localization and repair.<n>We localize faults through a series of steps, which include using Hoare logic to determine the state of each statement within the program.<n>Our tool achieves 89.6% fault localization coverage and GPT-4o mini yields the highest repair success rate of 74.18%.
  arXiv  Detail & Related papers  (2025-07-04T15:36:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.