Coding Agents Don't Know When to Act

• URL: http://arxiv.org/abs/2605.07769v1
• Date: Fri, 08 May 2026 14:10:00 GMT
• Title: Coding Agents Don't Know When to Act
• Abstract summary: Coding agents are increasingly deployed to autonomously maintain software.<n>In any real-world deployment, they will encounter stale bug reports about issues that have already been resolved.<n>We show that even state-of-the-art models fail, proposing undesirable changes in $35$ to $65%$ of cases.
• Score: 3.2610504259514754
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Coding agents are increasingly deployed to autonomously maintain software, including to resolve user-reported issues: a bug report comes in and the agent creates a patch to address it. However, in any real-world deployment, they will encounter stale bug reports about issues that have already been resolved. Agents should recognize this and abstain from modifying the code to avoid accumulating technical debt. To systematically evaluate whether current agents do so, we introduce FixedBench, a code benchmark with 200 human-verified coding tasks in which no code changes are required, testing five recent models across four agent harnesses. We find that even state-of-the-art models fail, proposing undesirable changes (excluding tests and documentation) in $35$ to $65\%$ of cases. Explicit instructions to reproduce the issue before patching partially address this issue but introduce a new failure mode: when an issue is partially fixed, they abstain even though a patch would still be needed. More broadly, our results indicate that LLMs fall prey to an action bias: they choose to act even if inaction would be appropriate. To break this pattern, inaction needs to be explicitly framed as a path to success, which highlights an overreliance on human guidance implicit in current training objectives.

Related papers

• TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization [57.14479960309637]
  Coding agents are increasingly used to accelerate code generation in many downstream tasks.<n>Despite their value, agent-generated code tends to be larger and more verbose than the corresponding human-written implementation.<n>We show that the cause lies in the agent's own search process.
  arXiv  Detail & Related papers  (2026-07-20T17:06:19Z)
• Writing Bug Reports for Software Repair Agents: What Information Matters Most? [45.10027008889631]
  We study what makes a bug report agent-ready.<n>We focus on the 441 issues representing bug reports.<n>Our results suggest that agentic-first reports benefit most from information that narrows the agent's search and repair space.
  arXiv  Detail & Related papers  (2026-07-10T16:00:57Z)
• Is Agent Code Less Maintainable Than Human Code? [17.226150835020146]
  We investigate how agent code compares to human code in maintenance settings.<n>We find that agents are less effective at resolving tasks when building on agent code compared to human code.
  arXiv  Detail & Related papers  (2026-06-19T23:56:14Z)
• Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset [0.0]
  We find that 46.41% of the fixes proposed by the agents Copilot, Devin, Cursor, and Claude are rejected.<n>Our goal in this paper is to understand the failure modes of AI-agents, an understanding that is crucial for better integrating AI-agents as efficient teammates.
  arXiv  Detail & Related papers  (2026-06-11T15:19:36Z)
• ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
  ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
  arXiv  Detail & Related papers  (2026-04-05T17:55:23Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• Red Teaming Program Repair Agents: When Correct Patches can Hide Vulnerabilities [22.02073334787359]
  We propose SWExploit, which generates adversarial issue statements to make APR agents produce patches that are functionally correct yet vulnerable.<n>Based on our evaluation, we are the first to challenge the traditional assumption that a patch passing all tests is inherently reliable and secure.
  arXiv  Detail & Related papers  (2025-09-30T07:38:57Z)
• Can Agents Fix Agent Issues? [15.50260831159089]
  LLM-based agent systems are emerging as a new software paradigm and have been widely adopted across diverse domains such as medicine, robotics, and programming.<n>Maintaining these systems requires substantial effort, as they are inevitably prone to bugs and continually evolve to meet changing external requirements.<n>Recent software engineering (SE) agents have shown promise in addressing issues in traditional software systems, but it remains unclear how effectively they can resolve real-world issues in agent systems.
  arXiv  Detail & Related papers  (2025-05-27T05:45:03Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)
• Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios [13.949319911378826]
  This study evaluated 4,892 patches from 10 top-ranked agents on 500 real-world GitHub issues.<n>No single agent dominated, with 170 issues unresolved, indicating room for improvement.<n>Most agents maintained code reliability and security, avoiding new bugs or vulnerabilities.<n>Some agents increased code complexity, many reduced code duplication and minimized code smells.
  arXiv  Detail & Related papers  (2024-10-16T11:33:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.