RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations

• URL: http://arxiv.org/abs/2605.26177v1
• Date: Mon, 25 May 2026 06:26:43 GMT
• Title: RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations
• Abstract summary: We introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified.<n>RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access.<n>We propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving.
• Score: 51.43574078961796
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Code agents are currently having skillful performance on repository-level software engineering benchmarks, but it remains unclear whether success on end-to-end tasks such as issue resolution truly reflects repository context reasoning, the ability to identify the task-relevant information across multiple files and reason over the relations among them. To investigate this question, we introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified that adopts perturbation as a diagnostic tool to increase the demand for context reasoning by transforming how the repository is exposed. First, RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access. RepoMirage-Extend further turns perturbation-targeted structural bottlenecks into explicit tasks beyond issue resolution, where the average performance declines from 66.8% in the original setting to 25.3%, indicating a significant deficiency in repository context reasoning. Further trajectory analysis reveals an exploration drift, where agents access broader repository context but fail to turn it into effective structure information. Motivated by this observation, we propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving, and show that explicit structural scaffolding yields notable gains. These results uncover an previously overlooked gap in repository context reasoning for code agents and suggest that stronger structure-aware methods are potential to improve them.

Related papers

• Adaptive Critical Token-Aware Retrieval for Repository-Level Code Generation [53.85136813868056]
  ACToR is an adaptive critical token-aware retrieval framework for repository-level code generation.<n>We show that ACToR consistently outperforms state-of-the-art methods on repository-level benchmarks.
  arXiv  Detail & Related papers  (2026-09-01T17:59:39Z)
• RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models [54.53236295237077]
  We introduce RepoReasoner, a benchmark for evaluating repository-level code reasoning.<n> Output Prediction measures fine-grained, stateful execution reasoning across files.<n>Call Chain Prediction evaluates high-level architectural dependency understanding under noisy context.
  arXiv  Detail & Related papers  (2026-07-28T17:12:41Z)
• DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection [16.007199928732614]
  We present DREA, a hypothesis-driven framework for repository-level vulnerability detection.<n>Goal-directed context acquisition is the primary source of detection improvement in this design.<n>To support evaluation, we construct RepoPairBench, a repository-grounded benchmark of validated Python vulnerability-fix pairs from real-world projects.
  arXiv  Detail & Related papers  (2026-07-15T04:49:05Z)
• Beyond Textual Repository Exploration: Dual-Modal Structural Reasoning for Agentic Issue Resolution [24.103118645295655]
  CouplingVIEW brings visual reasoning into repository exploration for issue-resolution agents.<n>VIEW consistently improves issue-resolution performance across different agent architectures and model families.
  arXiv  Detail & Related papers  (2026-07-02T09:23:48Z)
• Meta-Agent: From Task Descriptions to Verified Multi-Agent Systems [36.485119937780524]
  We present Meta-Agent, a two-phase framework that automatically constructs and executes specialized multi-agent systems.<n>We introduce a three-level error attribution mechanism that distinguishes local, upstream, and structural failures.<n>Experiments against strong multi-agent baselines and ablation studies demonstrate consistent improvements in task success rate, error recovery, and workflow stability.
  arXiv  Detail & Related papers  (2026-05-24T19:38:54Z)
• BUILD-AND-FIND: An Effort-Aware Protocol for Evaluating Agent-Managed Codebases [0.0]
  We introduce BUILD-AND-FIND, a protocol for evaluating whether downstream agents can recover intended choices from generated repositories.<n>For each task, a builder sees a hidden repository specification and creates a specification-traced multiple-choice question bank.<n>The protocol separates behavioral correctness from artifact-side recovery and reports recovery accuracy, repeatability, implementation coverage, and inspection effort.
  arXiv  Detail & Related papers  (2026-05-07T12:35:27Z)
• AgentV-RL: Scaling Reward Modeling with Agentic Verifier [63.55502685076245]
  Verifiers have been demonstrated to enhance LLM reasoning via test-time scaling (TTS)<n>We propose Agentic Verifier, a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process.<n>We show that Agentic Verifier yields consistent performance gains under both parallel and sequential TTS.
  arXiv  Detail & Related papers  (2026-04-17T12:27:36Z)
• Cycle-Consistent Search: Question Reconstructability as a Proxy Reward for Search Agent Training [80.20022221643414]
  Cycle-Consistent Search is a gold-supervision-free framework for training search agents.<n>We show that CCS achieves performance comparable to supervised baselines.<n>These results suggest that CCS provides a scalable training paradigm for training search agents in settings where gold supervision is unavailable.
  arXiv  Detail & Related papers  (2026-04-14T17:00:18Z)
• A Benchmark for Evaluating Repository-Level Code Agents with Intermediate Reasoning on Feature Addition Task [11.218318079376365]
  RACE-bench is a reasoning-augmented benchmark for evaluating code agents on feature addition tasks.<n> RACE-bench contains 528 real-world feature addition instances from 12 open-source repositories.<n>We evaluate three representative repository-level code agents on RACE-bench.
  arXiv  Detail & Related papers  (2026-03-27T11:58:47Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• ARC: Active and Reflection-driven Context Management for Long-Horizon Information Seeking Agents [9.76162701959422]
  ARC is a framework for systematically formulate context management.<n>It treats context as a dynamic internal reasoning state during execution.<n>It consistently outperforms passive context compression methods.
  arXiv  Detail & Related papers  (2026-01-17T12:17:50Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• RepoScope: Leveraging Call Chain-Aware Multi-View Context for Repository-Level Code Generation [16.544483144957407]
  RepoScope is a call chain-aware multi-view context for repository-level code generation.<n>We present a novel call chain prediction method that utilizes the repository's structural semantics to improve the identification of callees in the target function.<n>RepoScope outperforms state-of-the-art methods, achieving up to a 36.35% relative improvement in pass@1 scores.
  arXiv  Detail & Related papers  (2025-07-20T02:35:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.