Building an Internal Coding Agent at Zup: Lessons and Open Questions

• URL: http://arxiv.org/abs/2604.09805v1
• Date: Fri, 10 Apr 2026 18:28:59 GMT
• Title: Building an Internal Coding Agent at Zup: Lessons and Open Questions
• Abstract summary: Internal coding agents face a gap between prototype performance and production readiness.<n>We show that targeted tool design and layered safety guardrails improved agent reliability more than prompt engineering.<n>These findings suggest that the engineering decisions surrounding the model -- not the model itself -- determine whether a coding agent delivers real value in practice.
• Score: 1.6954927683791832
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Enterprise teams building internal coding agents face a gap between prototype performance and production readiness. The root cause is that technical model quality alone is insufficient -- tool design, safety enforcement, state management, and human trust calibration are equally decisive, yet underreported in the literature. We present CodeGen, an internal coding agent at Zup, and show that targeted tool design (e.g., string-replacement edits over full-file rewrites) and layered safety guardrails improved agent reliability more than prompt engineering, while progressive human oversight modes drove organic adoption without mandating trust. These findings suggest that the engineering decisions surrounding the model -- not the model itself -- determine whether a coding agent delivers real value in practice.

Related papers

• Self-Evolving Coding Agents [11.43197414613246]
  Self-evolving coding agents can inspect repositories, invoke tools, execute tests, debug failures, and generate patches.<n>We first define self-evolving coding agents and distinguish them from conventional coding agents and general self-evolving agents.<n>We find that executable feedback, repository-level context, and coding trajectories give software engineering a distinctive role as a natural domain for agent self-evolution.
  arXiv  Detail & Related papers  (2026-08-04T09:43:46Z)
• On Automated and Explainable Provenance of AI-Generated Code [56.818202904805446]
  We argue that the next generation of CodeGenAI tools must be built on a foundation of explainable provenance.<n>We describe a research program combining large-scale empirical studies with post-hoc causal and interpretability techniques.
  arXiv  Detail & Related papers  (2026-08-03T14:50:31Z)
• Cheap Code, Costly Judgment: A Case Study on Governable Agentic Software Engineering [8.368707686524482]
  We study how engineers organize architectures, tools, evidence, and feedback loops so that AI-mediated development remains inspectable, correctable, and maintainable.<n>We develop a theory of governance conversion, expressed as a process model explaining how high-velocity agentic implementation becomes governable.
  arXiv  Detail & Related papers  (2026-07-01T15:44:15Z)
• AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems [82.01232437066487]
  AgentX is a production-deployed multi-agent system that restructures this production function.<n>It autonomously generates, implements, evaluates, and learns from recommendation experiments.<n>AgentX orchestrates four tightly coupled stages in a closed loop.
  arXiv  Detail & Related papers  (2026-06-25T10:42:28Z)
• Is Agent Code Less Maintainable Than Human Code? [17.226150835020146]
  We investigate how agent code compares to human code in maintenance settings.<n>We find that agents are less effective at resolving tasks when building on agent code compared to human code.
  arXiv  Detail & Related papers  (2026-06-19T23:56:14Z)
• Agentic Software: How AI Agents Are Restructuring the Software Paradigm [0.0]
  We argue that the emergence of AI agents constitutes a fundamental restructuring of what software is.<n>In the former, code is the carrier of pre-written decision logic; in the latter, the agent itself is the decision logic.<n>We conclude with a four-stage roadmap toward self-evolving agent ecosystems.
  arXiv  Detail & Related papers  (2026-06-04T02:30:06Z)
• The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development? [80.24951682268332]
  We introduce the Meta-Agent Challenge (MAC), an evaluation framework designed to test the capacity of frontier models for autonomous agent development.<n>To ensure evaluation integrity, this framework is secured by multi-layer defenses against reward hacking.<n>We demonstrate that meta-agents rarely match human-engineered baseline policies, and the few that do are dominated by proprietary frontier models.
  arXiv  Detail & Related papers  (2026-06-03T04:58:17Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• AI-Generated Smells: An Analysis of Code and Architecture in LLM and Agent-Driven Development [1.980498913496519]
  This paper presents a systematic audit of technical debt in AI-generated software.<n>We show that AI does not eliminate flaws but rather introduces a distinct machine signature of defects.<n>We conclude that future progress depends on equipping agents with explicit architectural foresight to ensure the software they build is not just functional, but also maintainable.
  arXiv  Detail & Related papers  (2026-05-04T15:41:13Z)
• An Agentic Evaluation Framework for AI-Generated Scientific Code in PETSc [7.236134946837382]
  petscagent-bench is an agentic framework built on an agents-evaluating-agents paradigm.<n>It orchestrates a 14-evaluator pipeline across five scoring categories: correctness, performance, code quality, algorithmic appropriateness, and library-specific conventions.<n>We demonstrate the framework on a benchmark suite of realistic problems using the PETSc library for HPC.
  arXiv  Detail & Related papers  (2026-03-16T22:46:10Z)
• Secure Code Generation via Online Reinforcement Learning with Vulnerability Reward Model [60.60587869092729]
  Large language models (LLMs) are increasingly used in software development, yet their tendency to generate insecure code remains a major barrier to real-world deployment.<n>We propose SecCoderX, an online reinforcement learning framework for functionality-preserving secure code generation.
  arXiv  Detail & Related papers  (2026-02-07T07:42:07Z)
• Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization [69.36509281190662]
  Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck.<n>We consider using AI agents to automate this manual coding, and focus on the open question of optimal agent design.<n>We demonstrate that a simple agent framework consistently generates adaptation code that outperforms human-expert solutions.
  arXiv  Detail & Related papers  (2025-12-02T18:42:26Z)
• Training Language Models to Generate Quality Code with Program Analysis Feedback [66.0854002147103]
  Code generation with large language models (LLMs) is increasingly adopted in production but fails to ensure code quality.<n>We propose REAL, a reinforcement learning framework that incentivizes LLMs to generate production-quality code.
  arXiv  Detail & Related papers  (2025-05-28T17:57:47Z)
• Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? [60.84912551069379]
  We present the Code-Development Benchmark (Codev-Bench), a fine-grained, real-world, repository-level, and developer-centric evaluation framework. Codev-Agent is an agent-based system that automates repository crawling, constructs execution environments, extracts dynamic calling chains from existing unit tests, and generates new test samples to avoid data leakage.
  arXiv  Detail & Related papers  (2024-10-02T09:11:10Z)
• CodeAgent: Autonomous Communicative Agents for Code Review [12.163258651539236]
  This work introduces tool, a novel multi-agent Large Language Model (LLM) system for code review automation. CodeAgent incorporates a supervisory agent, QA-Checker, to ensure that all the agents' contributions address the initial review question. Results demonstrate CodeAgent's effectiveness, contributing to a new state-of-the-art in code review automation.
  arXiv  Detail & Related papers  (2024-02-03T14:43:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.