CodeCompass: Navigating the Navigation Paradox in Agentic Code Intelligence

• URL: http://arxiv.org/abs/2602.20048v1
• Date: Mon, 23 Feb 2026 16:58:37 GMT
• Title: CodeCompass: Navigating the Navigation Paradox in Agentic Code Intelligence
• Abstract summary: We identify the Navigation Paradox: agents perform poorly because navigation and retrieval are fundamentally distinct problems.<n>We demonstrate that graph-based structural navigation via Code--a Model Context Protocol server exposing dependency graphs--achieves 99.4% task completion on hidden-dependency tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern code intelligence agents operate in contexts exceeding 1 million tokens--far beyond the scale where humans manually locate relevant files. Yet agents consistently fail to discover architecturally critical files when solving real-world coding tasks. We identify the Navigation Paradox: agents perform poorly not due to context limits, but because navigation and retrieval are fundamentally distinct problems. Through 258 automated trials across 30 benchmark tasks on a production FastAPI repository, we demonstrate that graph-based structural navigation via CodeCompass--a Model Context Protocol server exposing dependency graphs--achieves 99.4% task completion on hidden-dependency tasks, a 23.2 percentage-point improvement over vanilla agents (76.2%) and 21.2 points over BM25 retrieval (78.2%).However, we uncover a critical adoption gap: 58% of trials with graph access made zero tool calls, and agents required explicit prompt engineering to adopt the tool consistently. Our findings reveal that the bottleneck is not tool availability but behavioral alignment--agents must be explicitly guided to leverage structural context over lexical heuristics. We contribute: (1) a task taxonomy distinguishing semantic-search, structural, and hidden-dependency scenarios; (2) empirical evidence that graph navigation outperforms retrieval when dependencies lack lexical overlap; and (3) open-source infrastructure for reproducible evaluation of navigation tools.

Related papers

• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• How Much Static Structure Do Code Agents Need? A Study of Deterministic Anchoring [10.106526629957655]
  LLM-based code agents navigate repositories through keyword search but miss the structural relationships that define how software actually works.<n>We investigate whether lightweight static analysis can provide deterministic anchors for these agents.<n>Our study identifies what we call the deterministic anchoring effect: static structure helps less by making agents "smarter" and more by making their navigation disciplined and reproducible.
  arXiv  Detail & Related papers  (2026-06-25T12:50:01Z)
• Do Agents Need Semantic Metadata? A Comparative Study in Agentic Data Retrieval [3.6202634353482352]
  The rise of Large Language Models (LLMs) capable of navigating the unstructured web raises a fundamental question: Is semantic metadata still necessary for agentic data discovery?<n>We present a comparative analysis of agentic data retrieval across two distinct environments: a Baseline Agent searching billions of open-web documents, and a Semantic Agent leveraging a corpus of 90 million datasets using schema.org.<n>The Semantic Agent excels at retrieving actionable data, achieving a 44.9% higher precision for metadata-rich registries and a 46.6% higher precision for pages with machine-readable downloads among its returned results.
  arXiv  Detail & Related papers  (2026-05-27T17:46:43Z)
• RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations [51.43574078961796]
  We introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified.<n>RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access.<n>We propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving.
  arXiv  Detail & Related papers  (2026-05-25T06:26:43Z)
• How to Interpret Agent Behavior [56.59836196946289]
  We introduce ACT*ONOMY, a taxonomy for describing and analyzing agent behavior at runtime.<n>By providing a shared vocabulary, ACT*ONOMY helps researchers, agent designers, and end users interpret agent behavior more consistently.
  arXiv  Detail & Related papers  (2026-05-13T14:52:40Z)
• AgentSearchBench: A Benchmark for AI Agent Search in the Wild [21.739444037726418]
  We introduce AgentSearchBench, a large-scale benchmark for agent search in the wild.<n>The benchmark formalizes agent search as retrieval and reranking problems under both executable task queries and high-level task descriptions.<n>We show that lightweight behavioral signals, including execution-aware probing, can substantially improve ranking quality.
  arXiv  Detail & Related papers  (2026-04-24T10:53:54Z)
• GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows [90.35728421223673]
  GTA-2 is a hierarchical benchmark for General Tool Agents (GTA)<n>Built on real-world authenticity, it leverages real user queries, deployed tools, and multimodal contexts.<n>Experiments reveal a pronounced capability cliff: while frontier models already struggle on atomic tasks, top models achieving only 14.39% success.
  arXiv  Detail & Related papers  (2026-04-17T05:36:00Z)
• Strategic Navigation or Stochastic Search? How Agents and Humans Reason Over Document Collections [37.38277822936901]
  We introduce MADQA, a benchmark of 2,250 human-authored questions grounded in 800 heterogeneous PDF documents.<n>We show that while the best agents can match human searchers in raw accuracy, they succeed on largely different questions and rely on brute-force search to compensate for weak strategic planning.<n>We release the dataset and evaluation harness to help facilitate the transition from brute-force retrieval to calibrated, efficient reasoning.
  arXiv  Detail & Related papers  (2026-03-12T17:11:22Z)
• Beyond Blame: Rethinking SZZ with Knowledge Graph Search [13.82629698836299]
  We present AgenticSZZ, the first approach to apply Temporal Knowledge Graphs (TKGs) to software evolution analysis.<n>We show that AgenticSZZ achieves F1-scores of 0.48 to 0.74, with statistically significant improvements over state-of-the-art by up to 27%.
  arXiv  Detail & Related papers  (2026-02-03T00:10:48Z)
• OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding [57.39403818250357]
  We introduce OctoBench, which benchmarks scaffold-aware instruction following in repository-grounded agentic coding.<n> OctoBench includes 34 environments and 217 tasks instantiated under three scaffold types, and is paired with 7,098 objective checklist items.<n>Experiments reveal a systematic gap between task-solving and scaffold-aware compliance, underscoring the need for training and evaluation.
  arXiv  Detail & Related papers  (2026-01-15T12:36:08Z)
• Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization [69.36509281190662]
  Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck.<n>We consider using AI agents to automate this manual coding, and focus on the open question of optimal agent design.<n>We demonstrate that a simple agent framework consistently generates adaptation code that outperforms human-expert solutions.
  arXiv  Detail & Related papers  (2025-12-02T18:42:26Z)
• Agent READMEs: An Empirical Study of Context Files for Agentic Coding [8.019313057979522]
  We study 2,303 agent context files from 1,925 repositories to characterize their structure, maintenance, and content.<n>We find that these files are not static documentation but complex, difficult-to-read artifacts that evolve like configuration code, maintained through frequent, small additions.<n>These findings indicate that while developers use context files to make agents functional, they provide few guardrails to ensure that agent-written code is secure or performant, highlighting the need for improved tooling and practices.
  arXiv  Detail & Related papers  (2025-11-17T02:18:55Z)
• InteractComp: Evaluating Search Agents With Ambiguous Queries [36.05005463045869]
  We introduce InteractComp, a benchmark designed to evaluate whether search agents can recognize query ambiguity and actively interact to resolve it during search.<n> Evaluation of 17 models reveals striking failure: the best model achieves only 13.73% accuracy despite 71.50% with complete context.<n>This stagnation, coupled with the immediate feedback inherent to search tasks, makes InteractComp a valuable resource for both evaluating and training interaction capabilities in search agents.
  arXiv  Detail & Related papers  (2025-10-28T17:35:54Z)
• InfoAgent: Advancing Autonomous Information-Seeking Agents [143.15973604285304]
  We introduce InfoAgent, a deep research agent powered by an innovative data synthesis pipeline and orchestrated web search tools.<n>With our methods, InfoAgent achieves 15.3% accuracy on BrowseComp, 29.2% on BrowseComp-ZH, and 40.4% on Xbench-DS.
  arXiv  Detail & Related papers  (2025-09-29T17:59:57Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Benchmarking Deep Search over Heterogeneous Enterprise Data [73.55304268238474]
  We present a new benchmark for evaluating a form of retrieval-augmented generation (RAG)<n>RAG requires source-aware, multi-hop reasoning over diverse, sparsed, but related sources.<n>We build it using a synthetic data pipeline that simulates business across product planning, development, and support stages.
  arXiv  Detail & Related papers  (2025-06-29T08:34:59Z)
• AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Search [58.98450205734779]
  Large language model (LLM) agents have demonstrated strong capabilities across diverse domains.<n>Existing agent search methods suffer from three major limitations.<n>We introduce a comprehensive framework to address these challenges.
  arXiv  Detail & Related papers  (2025-06-06T12:07:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.