AgentForge: Execution-Grounded Multi-Agent LLM Framework for Autonomous Software Engineering

• URL: http://arxiv.org/abs/2604.13120v1
• Date: Mon, 13 Apr 2026 13:51:13 GMT
• Title: AgentForge: Execution-Grounded Multi-Agent LLM Framework for Autonomous Software Engineering
• Abstract summary: We introduce execution-grounded verification as a first-class principle.<n>We instantiate this principle in AGENTFORGE, a multi-agent framework.<n>AGENTFORGEtoken achieves 40.0% resolution on SWE-BENCH Lite.
• Score: 3.2126925586839623
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models generate plausible code but cannot verify correctness. Existing multi-agent systems simulate execution or leave verification optional. We introduce execution-grounded verification as a first-class principle: every code change must survive sandboxed execution before propagation. We instantiate this principle in AGENTFORGE, a multi-agent framework where Planner, Coder, Tester, Debugger, and Critic agents coordinate through shared memory and a mandatory Docker sandbox. We formalize software engineering with LLMs as an iterative decision process over repository states, where execution feedback provides a stronger supervision signal than next-token likelihood. AGENTFORGE achieves 40.0\% resolution on SWE-BENCH Lite, outperforming single-agent baselines by 26--28 points. Ablations confirm that execution feedback and role decomposition each independently drive performance. The framework is open-source at https://github.com/raja21068/AutoCodeAI.

Related papers

• LMSM: LLM Security Framework Inspired by Linux Security Modules [56.93644694627783]
  Large language models (LLMs) are increasingly deployed with layered defenses, yet malicious prompts can still bypass them.<n>We present Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving.
  arXiv  Detail & Related papers  (2026-08-26T12:13:05Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents [61.184954205350984]
  ClawArena-Team is a benchmark of 41 multi-turn, multimodal, multi-agent scenarios spanning 258 evaluation rounds and 72 staged updates.<n>All scoring is execution-based with no LLM judge: an overall score -- the Subagent-Management Score (SMS) -- multiplies task correctness by a least-privilege and modality-routing factor.
  arXiv  Detail & Related papers  (2026-06-30T06:06:08Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation [55.947962672433675]
  ChipMATE is the first self-trained multi-agent framework for RTL generation.<n>Inspired by industrial practice, ChipMATE pairs a Verilog agent with a Python reference-model agent that mutually verify each other's outputs.<n>ChipMATE achieves 75.0% and 80.1% pass@1 on VerilogEval V2 with 4B and 9B base models.
  arXiv  Detail & Related papers  (2026-05-13T01:04:21Z)
• Deterministic Replay for AI Agent Systems [0.0]
  AI agent systems that couple large language models with external tools and APIs are inherently non-deterministic.<n>Existing observability platforms capture execution logs but cannot reproduce a run in isolation.<n>We present a developer-first CLI framework for deterministic replay of agent executions.
  arXiv  Detail & Related papers  (2026-04-30T12:00:45Z)
• KAIJU: An Executive Kernel for Intent-Gated Execution of LLM Agents [5.931389072124715]
  Tool-calling autonomous agents based on large language models using ReAct exhibit three limitations.<n>Recent work moves towards separating planning from execution but in each case model remains coupled to the execution mechanics.<n>We introduce a system-level abstraction for LLM agents which decouples the execution agent from the reasoning layer.
  arXiv  Detail & Related papers  (2026-03-31T21:38:28Z)
• AgentFactory: A Self-Evolving Framework Through Executable Subagent Accumulation and Reuse [14.413401094877122]
  Recent works on LLM-based agent self-evolution primarily record successful experiences as textual prompts or reflections.<n>We propose AgentFactory, a new self-evolution paradigm that preserves successful task solutions as executable subagent code.<n>Saved subagents are pure Python code with standardized documentation, enabling portability across any Python-capable system.
  arXiv  Detail & Related papers  (2026-03-18T17:58:25Z)
• ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads? [0.8749675983608171]
  We introduce ISO-Bench, a benchmark for coding agents to test their capabilities on real-world inference tasks.<n>We curated 54 tasks from merged pull requests with measurable performance improvements.
  arXiv  Detail & Related papers  (2026-02-23T08:37:53Z)
• AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management [47.49917373646469]
  Existing defenses treat bloated memory as given and focus on remaining resilient.<n>We present AgentSys, a framework that defends against indirect prompt injection through explicit memory management.
  arXiv  Detail & Related papers  (2026-02-07T06:28:51Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement [61.35824395228412]
  Large language model (LLM) based agents are increasingly used to tackle software engineering tasks.<n>We propose Self-Abstraction from Grounded Experience (SAGE), a framework that enables agents to learn from their own task executions.
  arXiv  Detail & Related papers  (2025-11-08T08:49:38Z)
• Where LLM Agents Fail and How They can Learn From Failures [62.196870049524364]
  Large Language Model (LLM) agents have shown promise in solving complex, multi-step tasks.<n>They amplify vulnerability to cascading failures, where a single root-cause error propagates through subsequent decisions.<n>Current systems lack a framework that can comprehensively understand agent error in a modular and systemic way.<n>We introduce the AgentErrorTaxonomy, a modular classification of failure modes spanning memory, reflection, planning, action, and system-level operations.
  arXiv  Detail & Related papers  (2025-09-29T18:20:27Z)
• CodeAgents: A Token-Efficient Framework for Codified Multi-Agent Reasoning in LLMs [16.234259194402163]
  We introduce CodeAgents, a prompting framework that codifies multi-agent reasoning and enables structured, token-efficient planning in multi-agent systems.<n>Results show consistent improvements in planning performance, with absolute gains of 3-36 percentage points over natural language prompting baselines.
  arXiv  Detail & Related papers  (2025-07-04T02:20:19Z)
• FlowReasoner: Reinforcing Query-Level Meta-Agents [63.602173107171076]
  This paper proposes a query-level meta-agent named FlowReasoner to automate the design of query-level multi-agent systems.<n>Our core idea is to incentivize a reasoning-based meta-agent via external execution feedback.
  arXiv  Detail & Related papers  (2025-04-21T17:35:42Z)
• A Unified Debugging Approach via LLM-Based Multi-Agent Synergy [39.11825182386288]
  FixAgent is an end-to-end framework for unified debug through multi-agent synergy. It significantly outperforms state-of-the-art repair methods, fixing 1.25$times$ to 2.56$times$ bugs on the repo-level benchmark, Defects4J.
  arXiv  Detail & Related papers  (2024-04-26T04:55:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.