SHERLOC: Structured Diagnostic Localization for Code Repair Agents

• URL: http://arxiv.org/abs/2606.24820v1
• Date: Tue, 23 Jun 2026 17:05:50 GMT
• Title: SHERLOC: Structured Diagnostic Localization for Code Repair Agents
• Abstract summary: We introduce SHERLOC, a training-free framework pairing a reasoning LLM with compact repository tools and self-recovery.<n>SHERLOC reaches state-of-the-art localization across model scales: 84.33% accuracy@1 on SWE-Bench Lite and 81.27% recall@1 on SWE-Bench Verified.
• Score: 33.35079971033214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM agents solve repository-level coding tasks through multi-turn tool use, but utilize half their budget on locating faults before editing. Dedicated localization frameworks have emerged, yet are still evaluated as file retrieval rather than actionable diagnosis, producing locations without the diagnostic context a repair agent needs. We introduce SHERLOC (Structured Hypothesis-driven Exploration and Reasoning for Localization), a training-free framework pairing a reasoning LLM with compact repository tools and self-recovery, without fine-tuning or multi-agent orchestration. SHERLOC reaches state-of-the-art localization across model scales: 84.33% accuracy@1 on SWE-Bench Lite and 81.27% recall@1 on SWE-Bench Verified; at ~30B parameters, it matches or outperforms other agentic methods. Injecting our locations and diagnostic findings into repair agents yields, on average, +5.95 pp resolve rate on SWE-Bench Verified while cutting localization and total tokens by 36.7% and 23.1%.

Related papers

• A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination [51.06539604709775]
  We propose A-SR, a self-evolving agentic framework that shifts the control unit from expression edits to role-conditioned evidence views.<n>A-SR coordinates formula discovery through routing among coordination protocols, an online evaluator-reward role policy, and state-routed process memory.
  arXiv  Detail & Related papers  (2026-08-05T14:01:10Z)
• AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents [83.37969790806069]
  LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.<n>We present Agent DebuggerX, an open-source framework that organizes debug as a closed loop of Detect, Attribute, Recover, and Rerun.<n>At its core, Deep Debugger performs multi-turn root-cause diagnosis through global trajectory understanding.
  arXiv  Detail & Related papers  (2026-07-21T06:21:13Z)
• LLM-as-a-Verifier: A General-Purpose Verification Framework [74.40111651545979]
  We introduce LLM-as-a-Verifier, a general-purpose verification framework.<n>It provides fine-grained feedback for agentic tasks without requiring additional training.<n>It achieves state-of-the-art performance on several benchmarks.
  arXiv  Detail & Related papers  (2026-07-06T17:59:35Z)
• Probe-and-Refine Tuning of Repository Guidance for Coding Agents [0.0]
  LLM-based coding agents need higher-level operational knowledge about a repository.<n>Recent studies disagree on whether LLM-generated guidance improves or harms agent performance.<n>In this paper we show that how the guidance is produced is the decisive variable.
  arXiv  Detail & Related papers  (2026-06-18T17:30:15Z)
• Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories [51.22051230894794]
  Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable.<n>We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through expert review.<n>We propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path.
  arXiv  Detail & Related papers  (2026-06-01T10:50:26Z)
• BLAgent: Agentic RAG for File-Level Bug Localization [2.2917707112773593]
  BLAgent is a novel agentic RAG framework for file-level bug localization.<n> BLAgent attains over 78% Top-1 accuracy with open-source models.<n> BLAgent improves end-to-end repair success by over 20%.
  arXiv  Detail & Related papers  (2026-05-18T07:20:13Z)
• Clover: A Neural-Symbolic Agentic Harness with Stochastic Tree-of-Thoughts for Verified RTL Repair [5.67664197679672]
  Clover orchestrates RTL repair as a structured search over code manipulations to explore a validated solution for the bug.<n>Clover fixes 96.8% of bugs within a fixed time limit, covering 94% and 63% more bugs than both pure traditional and LLM-based baselines.
  arXiv  Detail & Related papers  (2026-04-19T07:04:49Z)
• SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair [22.745971570878435]
  We propose a Suggestion-Guided multi-Agent framework for repository-level software repair.<n> SGAgent introduces a suggestion phase to strengthen the transition from localization to repair.<n>Three specialized sub-agents collaborate to achieve automated end-to-end software repair.
  arXiv  Detail & Related papers  (2026-02-27T03:32:47Z)
• Reformulate, Retrieve, Localize: Agents for Repository-Level Bug Localization [2.4063592468412276]
  Bug localization remains a critical yet time-consuming challenge in large-scale software repositories.<n>Recent advances in large language models (LLMs) have improved bug localization through query reformulation.<n>In this study, we investigate how an LLM-powered agent can improve file-level bug localization via lightweight query reformulation and summarization.
  arXiv  Detail & Related papers  (2025-12-07T22:25:11Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Alita-G: Self-Evolving Generative Agent for Agent Generation [54.49365835457433]
  We present ALITA-G, a framework that transforms a general-purpose agent into a domain expert.<n>In this framework, a generalist agent executes a curated suite of target-domain tasks.<n>It attains strong gains while reducing computation costs.
  arXiv  Detail & Related papers  (2025-10-27T17:59:14Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.