PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization

• URL: http://arxiv.org/abs/2607.19653v1
• Date: Wed, 22 Jul 2026 01:19:42 GMT
• Title: PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization
• Abstract summary: PerfAgent is a profiler-guided, verifier-in-the-loop workflow that gives an off-the-shelf coding agent the feedback needed to find real hotspots.<n>It more than doubles the rate of expert-matching patches over OpenHands with GPT-5.1, improving from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite.<n>It surpasses an oracle best-of-five baseline at substantially lower cost, showing that the gains come from better feedback rather than additional test-time sampling.
• Score: 8.134365862874384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents now perform well on correctness-oriented repository-level tasks, including SWE-Bench issue resolution and feature implementation in real codebases. However, they still struggle with repository-level code optimization, which requires preserving behavior while improving runtime performance. Passing tests is not enough in this setting; a patch must preserve behavior, implement code optimization, and approach expert speedups. Current agents often miss bottlenecks hidden behind abstraction layers and native extensions, stop after shallow speedups, or insufficiently test the code patches that thus may silently break edge cases. We present PerfAgent, a profiler-guided, verifier-in-the-loop workflow that gives an off-the-shelf coding agent the feedback needed to find real hotspots, improve beyond the first passing patch, and use profiler evidence rather than timing alone to decide what to optimize next. On two challenging optimization benchmarks, GSO and SWE-fficiency-Lite, PerfAgent more than doubles the rate of expert-matching patches over OpenHands with GPT-5.1, improving from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite. It also surpasses an oracle best-of-five baseline at substantially lower cost, showing that the gains come from better feedback rather than additional test-time sampling.

Related papers

• Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0 [53.71882250666667]
  Most reported gains from agent-optimization methods are one-shot.<n>This does not test the setting that matters for deployed agents.<n>We compare three approaches to agent-harness optimization.
  arXiv  Detail & Related papers  (2026-07-15T16:36:04Z)
• PERFOPT-Bench: Evaluating Coding Agents on Software Performance Optimization [18.924749191192433]
  Performance optimization is a distinct agentic task.<n>We introduce PERFOPT-Bench, a benchmark for evaluating this full performance-engineering loop.
  arXiv  Detail & Related papers  (2026-07-08T08:32:18Z)
• Can Coding Agents Implement Missed Compiler Optimizations? Evaluating LLM Agents on LLVM Peephole Optimizations [3.2479630553628156]
  We introduce PeepholeBench, an evaluation framework whose tasks are constructed from real-world missed peephole optimizations.<n>We benchmark state-of-the-art coding agents on fixing missed peephole optimizations in LLVM's InstCombine pass.<n>We observe a tension between correctness and profitability, and no agent matches human developers on both dimensions at once.
  arXiv  Detail & Related papers  (2026-07-02T18:25:29Z)
• REVES: REvision and VErification--Augmented Training for Test-Time Scaling [53.197756110943395]
  We propose a two-stage iterative framework that alternates between online data/prompt augmentation and policy optimization.<n>We observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi-turn training.
  arXiv  Detail & Related papers  (2026-06-17T10:37:23Z)
• AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation [59.964989458924585]
  AdaExplore is an agent framework that enables self-improvement via accumulated execution feedback for performance-critical kernel code generation.<n>In the adaptation stage, the agent synthesizes tasks and converts recurring failures into a reusable memory of validity rules.<n>In the search stage, the agent organizes candidate kernels as a tree and alternates between small local refinements and larger structural regeneration.
  arXiv  Detail & Related papers  (2026-04-17T18:25:03Z)
• ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads? [0.8749675983608171]
  We introduce ISO-Bench, a benchmark for coding agents to test their capabilities on real-world inference tasks.<n>We curated 54 tasks from merged pull requests with measurable performance improvements.
  arXiv  Detail & Related papers  (2026-02-23T08:37:53Z)
• SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads? [22.075705411944895]
  SWE-fficiency is a benchmark for evaluating repository-level performance optimization on real workloads.<n>Our suite contains 498 tasks across nine widely used data-science, machine-learning, and HPC repositories.
  arXiv  Detail & Related papers  (2025-11-08T17:55:09Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization [51.280919773837645]
  We develop ScoreFlow, a high-performance framework for agent workflow optimization.<n>ScoreFlow incorporates Score-DPO, a novel variant of the direct preference optimization method that accounts for quantitative feedback.<n>It achieves an 8.2% improvement over existing baselines across question answering, coding, and mathematical reasoning.
  arXiv  Detail & Related papers  (2025-02-06T18:47:49Z)
• Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement [47.89758553708932]
  We introduce textbfThinkCoder, a framework that combines thorough exploration with optimal refinement.<n>The exploration phase diversifies the solution space by searching for potential solutions, followed by a refinement phase that enhances precision.<n>To further minimize test-time computation overhead, we introduce preference-driven optimization with Reinforced Self-Training (ReST)
  arXiv  Detail & Related papers  (2024-12-30T07:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.