Writing Bug Reports for Software Repair Agents: What Information Matters Most?

• URL: http://arxiv.org/abs/2607.09553v1
• Date: Fri, 10 Jul 2026 16:00:57 GMT
• Title: Writing Bug Reports for Software Repair Agents: What Information Matters Most?
• Abstract summary: We study what makes a bug report agent-ready.<n>We focus on the 441 issues representing bug reports.<n>Our results suggest that agentic-first reports benefit most from information that narrows the agent's search and repair space.
• Score: 45.10027008889631
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Software development is increasingly moving toward agentic-first workflows. This includes AI agents responsible for generating initial fixes for submitted issue reports. In this setting, issue reports are no longer merely documentation for human maintainers; they become the primary task specification for the agent. However, little is known about how such reports should be written to maximize the agent's chances of producing a correct fix. We study what makes a bug report agent-ready. Starting from the SWE-bench Verified benchmark (i.e., a collection of 500 real repository issues with human-written gold patches and test suites for evaluating generated fixes) we manually classify each issue by change type (e.g., bug fix vs refactoring) and annotate each sentence with its information type, such as observed behavior, expected behavior, reproduction steps, localization cues, and suggested fixes. We focus on the 441 issues representing bug reports, and we run on them mini-swe-agent using three LLM backbones (i.e., GPT-5-mini, MiniMax M2.5, and Gemini 3 Flash). We then fit a binomial regression model to estimate the incremental association between each information type and agent success, controlling for confounding factors. Our results suggest that agentic-first reports benefit most from information that narrows the agent's search and repair space. Localization cues, such as references to affected code areas, are positively associated with successful repairs, while suggested fixes, expressed either in code or natural language, show some of the strongest positive associations with pass probability. An ablation study removing selected information types confirms that agents benefit less from information traditionally useful to humans, such as reproduction steps, and more from sentences that expose a repair direction, either through bug localization or a suggested fix.

Related papers

• Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research [75.8770212601511]
  Deep research (DR) systems produce long-form cited reports by orchestrating multiple agents that search and synthesize information from the web.<n> Citations are the primary mechanism for evaluating the faithfulness of these reports, yet current DR systems exhibit poor citation recall.<n>We propose an evaluation method that pinpoints which agent introduced each error by locally testing agent invocations for faithfulness and verifiability relative to their own inputs.
  arXiv  Detail & Related papers  (2026-08-25T09:34:18Z)
• What Makes a Good Bug Report for an AI Agent? [8.139494830308486]
  A good bug report for an agent overlaps with, but is not identical to, a good report for a human.<n> fix suggestions, reproduction scripts, repository source code, and localization info are associated with higher resolution likelihood.<n>Some qualities long emphasized for human readers, such as natural language steps to reproduce and readable descriptions, contribute little or even correlate with lower success.
  arXiv  Detail & Related papers  (2026-07-08T16:13:15Z)
• ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues [54.98461672730087]
  We introduce ReproRepo, a scalable framework for evaluation that leverages human-raised GitHub issues as naturally occurring supervision on realistic reproduction blockers.<n>We instantiate ReproRepo on 1,149 recent machine learning papers from major conferences and evaluate four frontier model-agent configurations.<n>Our results show that LLM agents, even without executing code, can identify many real-world problems from paper-repository pairs.
  arXiv  Detail & Related papers  (2026-06-16T17:58:05Z)
• Coding Agents Don't Know When to Act [3.2610504259514754]
  Coding agents are increasingly deployed to autonomously maintain software.<n>In any real-world deployment, they will encounter stale bug reports about issues that have already been resolved.<n>We show that even state-of-the-art models fail, proposing undesirable changes in $35$ to $65%$ of cases.
  arXiv  Detail & Related papers  (2026-05-08T14:10:00Z)
• ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
  ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
  arXiv  Detail & Related papers  (2026-04-05T17:55:23Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• Where LLM Agents Fail and How They can Learn From Failures [62.196870049524364]
  Large Language Model (LLM) agents have shown promise in solving complex, multi-step tasks.<n>They amplify vulnerability to cascading failures, where a single root-cause error propagates through subsequent decisions.<n>Current systems lack a framework that can comprehensively understand agent error in a modular and systemic way.<n>We introduce the AgentErrorTaxonomy, a modular classification of failure modes spanning memory, reflection, planning, action, and system-level operations.
  arXiv  Detail & Related papers  (2025-09-29T18:20:27Z)
• Crash Report Enhancement with Large Language Models: An Empirical Study [21.842377922082104]
  We examine whether large language models can enhance crash reports by adding fault locations, root-cause explanations, and repair suggestions.<n>On a dataset of 492 real-world crash reports, LLM-enhanced reports improve Top-1 problem-localization accuracy from 10.6% to 40.2-43.1%.<n>Both our manual evaluations and LLM-as-a-judge assessment show that Agentic-LLM delivers stronger root-cause explanations and more actionable repair guidance.
  arXiv  Detail & Related papers  (2025-09-16T21:02:57Z)
• Evaluating Agent-based Program Repair at Google [9.62742759337993]
  Agent-based program repair offers to automatically resolve complex bugs end-to-end.<n>Recent work has explored the use of agent-based repair approaches on the popular open-source SWE-Bench.<n>This paper explores the viability of using an agentic approach to address bugs in an enterprise context.
  arXiv  Detail & Related papers  (2025-01-13T18:09:25Z)
• On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents [58.79302663733703]
  Large language model-based multi-agent systems have shown great abilities across various tasks due to the collaboration of expert agents.<n>The impact of clumsy or even malicious agents--those who frequently make errors in their tasks--on the overall performance of the system remains underexplored.<n>This paper investigates what is the resilience of various system structures under faulty agents on different downstream tasks.
  arXiv  Detail & Related papers  (2024-08-02T03:25:20Z)
• Automatic Classification of Bug Reports Based on Multiple Text Information and Reports' Intention [37.67372105858311]
  This paper proposes a new automatic classification method for bug reports. The innovation is that when categorizing bug reports, in addition to using the text information of the report, the intention of the report is also considered. Our proposed method achieves better performance and its F-Measure achieves from 87.3% to 95.5%.
  arXiv  Detail & Related papers  (2022-08-02T06:44:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.