Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study
- URL: http://arxiv.org/abs/2608.01507v1
- Date: Sun, 02 Aug 2026 21:36:59 GMT
- Title: Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study
- Abstract summary: We compare the two approaches on SWE-QA, a benchmark for repository-level code question answering.<n>Semantic search answered 65.2% of questions correctly against 46.2% for deep agentic search.<n>Deep agentic search addresses a real problem and is now the preferred design in many code agents.
- Score: 0.18737506366172094
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Code agents spend much of their effort simply locating the right code inside a repository. Two approaches dominate current practice. In Semantic Search, the agent retrieves code blocks from a vector index built from the repository in advance. In Deep Agentic Search (also known as grep-search by subagent), a planning agent delegates the exploration to a separate subagent that works in an isolated context window and returns only a condensed result. The second design, which is considered good context engineering practice, exists to protect the main agent from context pollution (also known as context rot), the loss of accuracy that occurs as unrelated material accumulates in the context window. Recent code agents (such as Claude Code, Codex, Antigravity, etc) have adopted it quickly, but there is little evidence on whether it produces better answers. We compare the two approaches on SWE-QA, a benchmark for repository-level code question answering. Semantic search answered 65.2% of questions correctly against 46.2% for deep agentic search, and it produced each correct answer at less than half the cost. To explain the gap, we then coded every failed run into a taxonomy of failure modes. The taxonomy shows that deep agentic search did not remove failures but introduced a new class of them: the single largest share of its failures, 41.8%, occurred at the hand-off between the planner and its sub-agent, and these were usually silent, ending in a fluent and confident answer that was wrong. Deep agentic search addresses a real problem and is now the preferred design in many code agents. However, our results show that the protection it offers may not be free, and that for read-only questions over a repository that can be indexed, retrieval was the stronger and cheaper option.
Related papers
- DeepRepoQA: Code Repository Question Answering with Deep Agent Exploration [10.655923413185498]
DeepRepoQA is a novelQA framework for repository-level code understanding.<n>A Monte-Carlo Tree Search (MCTS) mechanism is employed to empower agents to dynamically search, navigate, and inspect code.
arXiv Detail & Related papers (2026-08-25T08:26:31Z) - The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows [72.9051182568928]
We present ReASearch, a unified framework for reasoning-driven optimization.<n>With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML.<n>We observe that complex search behaviors emerge naturally from the agent's reasoning process.
arXiv Detail & Related papers (2026-08-07T02:15:39Z) - Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories [0.0]
Persistent context files (AGENTS.md, CLAUDE.md) are standard practice for guiding AI coding agents, yet evidence for their effectiveness is contradictory.<n>We present a controlled ablation of context-injection strategy across two frontier agents (Claude Code and Codex)
arXiv Detail & Related papers (2026-07-28T11:07:53Z) - FastContext: Training Efficient Repository Explorer for Coding Agents [29.359764163872473]
Large Language Model (LLM) coding agents have achieved strong results on software engineering tasks.<n> repository exploration remains a major bottleneck: locating relevant code consumes substantial token budget and pollutes the agent's context with irrelevant snippets.<n>We present FastContext, a dedicated exploration subagent that separates repository exploration from solving.
arXiv Detail & Related papers (2026-06-12T03:23:59Z) - SWE-Explore: Benchmarking How Coding Agents Explore Repositories [20.73123882302075]
Repository-level coding benchmarks such as SWE-bench have driven a rapid surge in the capabilities of coding agents.<n>In this paper, we introduce SWE-Explore, a benchmark that isolates the evaluation of repository exploration.
arXiv Detail & Related papers (2026-06-05T14:08:27Z) - Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories [51.22051230894794]
Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable.<n>We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through expert review.<n>We propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path.
arXiv Detail & Related papers (2026-06-01T10:50:26Z) - RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations [51.43574078961796]
We introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified.<n>RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access.<n>We propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving.
arXiv Detail & Related papers (2026-05-25T06:26:43Z) - Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction [127.64173950476702]
We study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general-purpose terminal tools.<n>This approach requires no offline indexing and adapts naturally to evolving local corpora.<n>Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines.
arXiv Detail & Related papers (2026-05-03T19:13:11Z) - AgentSearchBench: A Benchmark for AI Agent Search in the Wild [21.739444037726418]
We introduce AgentSearchBench, a large-scale benchmark for agent search in the wild.<n>The benchmark formalizes agent search as retrieval and reranking problems under both executable task queries and high-level task descriptions.<n>We show that lightweight behavioral signals, including execution-aware probing, can substantially improve ranking quality.
arXiv Detail & Related papers (2026-04-24T10:53:54Z) - SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks [55.76734816061826]
We introduce SlopCodeBench, a language-agnostic benchmark comprising 20 problems and 93 checkpoints.<n>We track two trajectory-level quality signals: verbosity, the fraction of redundant or duplicated code, and structural erosion.<n>No agent solves any problem end-to-end across 11 models; the highest checkpoint solve rate is 17.2%.
arXiv Detail & Related papers (2026-03-25T19:26:44Z) - HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches [54.65565885083031]
We propose a hierarchical agentic deep search framework, HierSearch, trained with hierarchical RL.<n>At the low level, a local deep search agent and a Web deep search agent are trained to retrieve evidence from their corresponding domains.<n>At the high level, a planner agent coordinates low-level agents and provides the final answer.
arXiv Detail & Related papers (2025-08-11T15:31:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.