Archer: Towards Agentic Review for Compiler Optimizations

• URL: http://arxiv.org/abs/2607.01808v1
• Date: Thu, 02 Jul 2026 07:22:40 GMT
• Title: Archer: Towards Agentic Review for Compiler Optimizations
• Abstract summary: We present Archer, the first automated agentic code review tool for compiler optimizations.<n>We evaluate Archer on 70 open PRs and 328 closed PRs in LLVM from the last two months.
• Score: 2.4796712345498015
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern compilers are frequently updated, but expert review capacity is highly limited, leading to delayed integration and, in some cases, subtle semantic bugs entering the compiler codebase. Automating the code review process with modern general code review agents may be feasible, but it faces critical challenges due to compiler complexity. In this paper, we use LLVM as our target compiler and present Archer, the first automated agentic code review tool for compiler optimizations. Archer constrains the agentic review process from both ends by using obligations to guide analysis and a deterministic validation guard to admit only findings backed by executable evidence. We evaluated Archer on 70 open PRs and 328 closed PRs in LLVM from the last two months. The review results are shocking and concerning: Archer discovers that 21% of open PRs and 11% of closed PRs are buggy, i.e, introducing semantic bugs such as miscompilations in LLVM. Our findings expose a critical gap in the capacity for critical review in large compiler projects and demonstrate the practical value of Archer as an additional reviewer.

Related papers

• Can Coding Agents Implement Missed Compiler Optimizations? Evaluating LLM Agents on LLVM Peephole Optimizations [3.2479630553628156]
  We introduce PeepholeBench, an evaluation framework whose tasks are constructed from real-world missed peephole optimizations.<n>We benchmark state-of-the-art coding agents on fixing missed peephole optimizations in LLVM's InstCombine pass.<n>We observe a tension between correctness and profitability, and no agent matches human developers on both dimensions at once.
  arXiv  Detail & Related papers  (2026-07-02T18:25:29Z)
• Understanding Agent-Based Patching of Compiler Missed Optimizations [22.677231886742547]
  We identify a significant challenge that patching a missed optimization requires more than just fixing the reported case.<n>We construct a benchmark of real-world LLVM missed optimization issues and compare agent-generated patches with patches from developers in terms of optimization scope.
  arXiv  Detail & Related papers  (2026-07-02T16:12:32Z)
• AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference [58.285369364699]
  We present AgentCompile, an LLM-guided inference compiler that uses LLM outputs only as advisory search metadata.<n>Given compiler-derived region summaries and bounded candidate spaces, the LLM proposes semantic labels, candidate priorities, parameter hints, and risk annotations.<n>AgentCompile materializes candidates through graphs templates, checks interface and hardware constraints, validates candidates empirically, selects implementations by measured latency, and falls back when specialization is unsupported or unprofitable.<n>In end-to-end autoregressive generation, AgentCompile averages 5.66x, 4.05x, and 4.26x speedup over PyTorch on Qwen3-1.7B, Qwen
  arXiv  Detail & Related papers  (2026-06-04T03:49:56Z)
• Decaf: Improving Neural Decompilation with Automatic Feedback and Search [64.02615739798277]
  High-level syntax, identifiers, and custom data types are generally lost as the compiler translates human-readable code to low-level machine code.<n>Generative AI models are a natural fit for reconstructing high-level syntax, identifiers, and types.<n>We argue that compiler feedback can be used to dramatically improve the semantic correctness of neural decompiler outputs via search.
  arXiv  Detail & Related papers  (2026-05-12T04:21:26Z)
• SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks [55.76734816061826]
  We introduce SlopCodeBench, a language-agnostic benchmark comprising 20 problems and 93 checkpoints.<n>We track two trajectory-level quality signals: verbosity, the fraction of redundant or duplicated code, and structural erosion.<n>No agent solves any problem end-to-end across 11 models; the highest checkpoint solve rate is 17.2%.
  arXiv  Detail & Related papers  (2026-03-25T19:26:44Z)
• Agentic Harness for Real-World Compilers [22.07908620753322]
  We introduce the agentic harness designed to assist large compiler agents in understanding and fixing compiler bugs.<n>Our focus is on LLVM, one of the most widely used compiler infrastructures. Central to-autofix are agent-friendly LLVM tools, and a tailored minimal agent-autofix-mini for fixing LLVM bugs.
  arXiv  Detail & Related papers  (2026-03-20T15:56:43Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• Counting Without Running: Evaluating LLMs' Reasoning About Code Complexity [2.7389338551082605]
  We develop a benchmark to test Large Language Models (LLMs) for anticipating performance bottlenecks.<n>FLOPBench predicts single and double-precision FLOP counts for 577 kernels.<n>Our results positionFLOPBench as a focused testbed for developing LLM tooling.
  arXiv  Detail & Related papers  (2025-12-04T01:03:20Z)
• InspectCoder: Dynamic Analysis-Enabled Self Repair through interactive LLM-Debugger Collaboration [71.18377595277018]
  Large Language Models (LLMs) frequently generate buggy code with complex logic errors that are challenging to diagnose.<n>We present InspectCoder, the first agentic program repair system that empowers LLMs to actively conduct dynamic analysis via interactive debugger control.
  arXiv  Detail & Related papers  (2025-10-21T06:26:29Z)
• Fun with flags: How Compilers Break and Fix Constant-Time Code [0.0]
  We analyze how compiler optimizations break constant-time code.<n>Key insight is that a small set of passes are at the root of most leaks.<n>We propose an original and practical mitigation that requires no source code modification or custom compiler.
  arXiv  Detail & Related papers  (2025-07-08T15:52:17Z)
• CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System [52.048087777953064]
  We propose CompileAgent, an agent framework dedicated to repo-level compilation.<n>CompileAgent integrates five tools and a flow-based agent strategy, enabling interaction with software artifacts for compilation instruction search and error resolution.<n>We show that our method significantly improves the compilation success rate, ranging from 10% to 71%.
  arXiv  Detail & Related papers  (2025-05-07T08:59:14Z)
• RAG-Based Fuzzing of Cross-Architecture Compilers [0.8302146576157498]
  OneAPI is an open standard that supports cross-architecture software development with minimal effort from developers.<n>OneAPI brings DPC++ and C++ compilers which need to be thoroughly tested to verify their correctness, reliability, and security.<n>This paper proposes a large-language model (LLM)-based compiler fuzzing tool that integrates the concept of retrieval-augmented generation (RAG)
  arXiv  Detail & Related papers  (2025-04-11T20:46:52Z)
• Finding Missed Code Size Optimizations in Compilers using LLMs [1.90019787465083]
  We develop a novel testing approach which combines large language models with a series of differential testing strategies.<n>Our approach requires fewer than 150 lines of code to implement.<n>To date we have reported 24 confirmed bugs in production compilers.
  arXiv  Detail & Related papers  (2024-12-31T21:47:46Z)
• Dcc --help: Generating Context-Aware Compiler Error Explanations with Large Language Models [53.04357141450459]
  dcc --help was deployed to our CS1 and CS2 courses, with 2,565 students using the tool over 64,000 times in ten weeks. We found that the LLM-generated explanations were conceptually accurate in 90% of compile-time and 75% of run-time cases, but often disregarded the instruction not to provide solutions in code.
  arXiv  Detail & Related papers  (2023-08-23T02:36:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.