Update from Hell: Can Coding Agents Survive Hidden Breakage in Dependency Upgrades?

• URL: http://arxiv.org/abs/2608.30300v1
• Date: Mon, 31 Aug 2026 06:13:51 GMT
• Title: Update from Hell: Can Coding Agents Survive Hidden Breakage in Dependency Upgrades?
• Abstract summary: DEPBENCH is a benchmark consisting of 203 real-world dependency-upgrade tasks across five package ecosystems.<n>We evaluate mainstream coding agents on DEPBENCH, highlighting an important gap between current agent capabilities and real-world software maintenance needs.
• Score: 46.693059419882694
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern software systems rely heavily on third-party dependencies, but upgrading those dependencies remains a costly maintenance activity. Dependency upgrades do not always preserve the function signatures, type systems, APIs, or runtime semantics assumed by existing code. Consequently, developers often need to perform source code adaptations to accommodate dependency-induced changes. However, such code-level changes are often not explicitly communicated to project maintainers, posing a significant challenge to software reliability. Meanwhile, coding agents have emerged as a new form of software development tool and are increasingly adopted by developers due to their automation capabilities. In this paper, we introduce DEPBENCH, a benchmark consisting of 203 real-world dependency-upgrade tasks across five package ecosystems spanning five language communities, each involving hidden code-level changes that require source code adaptation. We evaluate mainstream coding agents on DEPBENCH. The best completed configuration solves only 104/203 tasks (51.2%), with substantial variation across agent harnesses, models, and ecosystems, highlighting an important gap between current agent capabilities and real-world software maintenance needs.

Related papers

• Agents in the Large: Perception-Centered Architecture for Persistent Agents [66.30987129595137]
  We propose a Perception-Centered Architecture for Persistent Agents (Pera)<n>Pera describes a persistent agent organized around perception and control components that continually perceive service-relevant signals.<n>We use Pera to retrospectively organize recent work, examine a detailed case study, and offer forward-looking insights for building more capable persistent agents.
  arXiv  Detail & Related papers  (2026-08-31T09:04:52Z)
• Self-Evolving Coding Agents [11.43197414613246]
  Self-evolving coding agents can inspect repositories, invoke tools, execute tests, debug failures, and generate patches.<n>We first define self-evolving coding agents and distinguish them from conventional coding agents and general self-evolving agents.<n>We find that executable feedback, repository-level context, and coding trajectories give software engineering a distinctive role as a natural domain for agent self-evolution.
  arXiv  Detail & Related papers  (2026-08-04T09:43:46Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development [72.4729759618632]
  We introduce ABC-Bench, a benchmark to evaluate agentic backend coding within a realistic, executable workflow.<n>We curated 224 practical tasks spanning 8 languages and 19 frameworks from open-source repositories.<n>Our evaluation reveals that even state-of-the-art models struggle to deliver reliable performance on these holistic tasks.
  arXiv  Detail & Related papers  (2026-01-16T08:23:52Z)
• Architecting AgentOps Needs CHANGE [0.0]
  We argue that architecting Agentic AI systems requires a shift from managing control loops to enabling dynamic co-evolution among agents, infrastructure, and human oversight.<n>We introduce a conceptual framework comprising six capabilities for operationalizing Agentic AI systems: Contextualize, Harmonize, Anticipate, Negotiate, Generate, and Evolve.
  arXiv  Detail & Related papers  (2026-01-10T06:51:39Z)
• NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents [79.29376673236142]
  Existing benchmarks fail to rigorously evaluate the long-horizon capabilities required to build complete software systems.<n>We present NL2Repo Bench, a benchmark explicitly designed to evaluate the long-horizon repository generation ability of coding agents.
  arXiv  Detail & Related papers  (2025-12-14T15:12:13Z)
• LLM Agents for Automated Dependency Upgrades [14.958375551367132]
  We introduce a framework of LLM agents to automatically recommend and apply code updates and ensure compatibility with new versions.<n>Our solution can automatically localize updated library usages in live Javas and implement recommended fixes in a user-friendly manner.<n>Results show that our approach not only performs upgrades using fewer tokens but also achieves a precision of 71.4%, highlighting its efficiency and effectiveness compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-03T19:57:10Z)
• Code2MCP: Transforming Code Repositories into MCP Services [53.234097255779744]
  Model Context Protocol (MCP) aims to create a standard for how Large Language Models use tools.<n>We introduce Code2MCP, an agent-based framework that automatically transforms a GitHub repository into a functional MCP service.<n>We demonstrate that Code2MCP successfully transforms open-source computing libraries in scientific fields such as bioinformatics, mathematics, and fluid dynamics.
  arXiv  Detail & Related papers  (2025-09-07T06:13:25Z)
• MaintainCoder: Maintainable Code Generation Under Dynamic Requirements [35.230289807466924]
  maintainability is a critical dimension in real-world software development.<n>We propose MaintainCoder to handle dynamic requirements with minimal rework.<n>It integrates the Waterfall model, design patterns, and multi-agent collaboration.
  arXiv  Detail & Related papers  (2025-03-31T16:06:47Z)
• Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios [13.949319911378826]
  This study evaluated 4,892 patches from 10 top-ranked agents on 500 real-world GitHub issues.<n>No single agent dominated, with 170 issues unresolved, indicating room for improvement.<n>Most agents maintained code reliability and security, avoiding new bugs or vulnerabilities.<n>Some agents increased code complexity, many reduced code duplication and minimized code smells.
  arXiv  Detail & Related papers  (2024-10-16T11:33:57Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.