The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering

• URL: http://arxiv.org/abs/2507.15003v1
• Date: Sun, 20 Jul 2025 15:15:58 GMT
• Title: The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering
• Abstract summary: This paper introduces AIDev, the first largescale dataset capturing how such agents operate in the wild.<n>Spanning over 456,000 pull requests by five leading agents, AIDev provides an unprecedented empirical foundation for studying autonomous teammates in software development.<n>The dataset includes rich on PRs, authorship, review timelines, code changes, and integration outcomes.
• Score: 10.252332355171237
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The future of software engineering--SE 3.0--is unfolding with the rise of AI teammates: autonomous, goal-driven systems collaborating with human developers. Among these, autonomous coding agents are especially transformative, now actively initiating, reviewing, and evolving code at scale. This paper introduces AIDev, the first large-scale dataset capturing how such agents operate in the wild. Spanning over 456,000 pull requests by five leading agents--OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code--across 61,000 repositories and 47,000 developers, AIDev provides an unprecedented empirical foundation for studying autonomous teammates in software development. Unlike prior work that has largely theorized the rise of AI-native software engineering, AIDev offers structured, open data to support research in benchmarking, agent readiness, optimization, collaboration modeling, and AI governance. The dataset includes rich metadata on PRs, authorship, review timelines, code changes, and integration outcomes--enabling exploration beyond synthetic benchmarks like SWE-bench. For instance, although agents often outperform humans in speed, their PRs are accepted less frequently, revealing a trust and utility gap. Furthermore, while agents accelerate code submission--one developer submitted as many PRs in three days as they had in three years--these are structurally simpler (via code complexity metrics). We envision AIDev as a living resource: extensible, analyzable, and ready for the SE and AI communities. Grounding SE 3.0 in real-world evidence, AIDev enables a new generation of research into AI-native workflows and supports building the next wave of symbiotic human-AI collaboration. The dataset is publicly available at https://github.com/SAILResearch/AI_Teammates_in_SE3. > AI Agent, Agentic AI, Coding Agent, Agentic Coding, Software Engineering Agent

Related papers

• SWE-chat: Coding Agent Interactions From Real Users in the Wild [70.18158706281724]
  SWE-chat is the first large-scale dataset of real coding agent sessions collected from open-source developers in the wild.<n>The dataset currently contains 6,000 sessions, comprising more than 63,000 user prompts and 355,000 agent tool calls.
  arXiv  Detail & Related papers  (2026-04-22T17:08:19Z)
• AI Co-Scientist for Ranking: Discovering Novel Search Ranking Models alongside LLM-based AI Agents with Cloud Computing Access [60.37646376282544]
  We present an AI Co-Scientist framework that automates the full search ranking research pipeline.<n>This is the first study in the ranking community to utilize an AI Co-Scientist framework for algorithmic research.
  arXiv  Detail & Related papers  (2026-03-23T09:29:08Z)
• AIDev: Studying AI Coding Agents on GitHub [9.1677959533705]
  We introduce AIDev, a large-scale dataset focused on agent-authored pull requests (Agentic-PRs) in real-world GitHub repositories.<n> AIDev aggregates 932,791 Agentic-PRs produced by five agents: OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code.<n>In addition, AIDev includes a curated subset of 33,596 Agentic-PRs from 2,807 repositories with over 100 stars, providing further information such as comments, reviews, commits, and related issues.
  arXiv  Detail & Related papers  (2026-02-09T20:45:58Z)
• AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents [49.67355440164857]
  We introduce AIRS-Bench, a suite of 20 tasks sourced from state-of-the-art machine learning papers.<n>Airs-Bench tasks assess agentic capabilities over the full research lifecycle.<n>We open-source the AIRS-Bench task definitions and evaluation code to catalyze further development in autonomous scientific research.
  arXiv  Detail & Related papers  (2026-02-06T16:45:02Z)
• On Autopilot? An Empirical Study of Human-AI Teaming and Review Practices in Open Source [11.412808537439973]
  We investigated project-level guidelines and developers' interactions with AI-assisted pull requests (PRs)<n>We found that over 67.5% of AI-co-authored PRs originate from contributors without prior code ownership.<n>In contrast to human-created PRs where non-owner developers receive the most feedback, AI-co-authored PRs from non-owners receive the least.
  arXiv  Detail & Related papers  (2026-01-20T09:09:53Z)
• Agentic AI for Software: thoughts from Software Engineering community [9.966138715949205]
  At the code level, common software tasks include code generation, testing, and program repair.<n>Key to successfully developing agentic AI-based software will be to resolve the core difficulty in software engineering - the deciphering and clarification of developer intent.<n>A successful deployment of agentic technology into software engineering would involve making conceptual progress in such intent inference via agents.
  arXiv  Detail & Related papers  (2025-08-24T12:57:21Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [66.1850490474361]
  We conduct the first academic study to explore developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants, GitHub Copilot and OpenHands.<n>Our results show agents have the potential to assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam? [51.112225746095746]
  We introduce X-Master, a tool-augmented reasoning agent designed to emulate human researchers.<n>X-Masters sets a new state-of-the-art record on Humanity's Last Exam with a score of 32.1%.
  arXiv  Detail & Related papers  (2025-07-07T17:50:52Z)
• Unified Software Engineering agent as AI Software Engineer [14.733475669942276]
  Large Language Model (LLM) technology has raised expectations for automated coding.<n>In this paper, we seek to understand this question by developing a Unified Software Engineering agent or USEagent.<n>We envision USEagent as the first draft of a future AI Software Engineer which can be a team member in future software development teams involving both AI and humans.
  arXiv  Detail & Related papers  (2025-06-17T16:19:13Z)
• R&D-Agent: Automating Data-Driven AI Solution Building Through LLM-Powered Automated Research, Development, and Evolution [60.80016554091364]
  R&D-Agent is a dual-agent framework for iterative exploration.<n>The Researcher agent uses performance feedback to generate ideas, while the Developer agent refines code based on error feedback.<n>R&D-Agent is evaluated on MLE-Bench and emerges as the top-performing machine learning engineering agent.
  arXiv  Detail & Related papers  (2025-05-20T06:07:00Z)
• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks [52.46737975742287]
  We introduce TheAgentCompany, a benchmark for evaluating AI agents that interact with the world in similar ways to those of a digital worker.<n>We find that the most competitive agent can complete 30% of tasks autonomously.<n>This paints a nuanced picture on task automation with simulating LM agents in a setting a real workplace.
  arXiv  Detail & Related papers  (2024-12-18T18:55:40Z)
• ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams [1.3967206132709542]
  ChatCollab's novel architecture allows agents - human or AI - to join collaborations in any role.<n>Using software engineering as a case study, we find that our AI agents successfully identify their roles and responsibilities.<n>In relation to three prior multi-agent AI systems for software development, we find ChatCollab AI agents produce comparable or better software in an interactive game development task.
  arXiv  Detail & Related papers  (2024-12-02T21:56:46Z)
• Improving Performance of Commercially Available AI Products in a Multi-Agent Configuration [11.626057561212694]
  Crowdbotics PRD AI is a tool for generating software requirements using AI. GitHub Copilot is an AI pair-programming tool. By sharing business requirements from PRD AI, we improve the code suggestion capabilities of GitHub Copilot by 13.8% and developer task success rate by 24.5%.
  arXiv  Detail & Related papers  (2024-10-29T15:28:19Z)
• Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap [30.996760992473064]
  Software Engineering 3.0 (SE 3.0) is an AI-native approach characterized by intent-first, conversation-oriented development. We outline the key components of the SE 3.0 technology stack, which includes Teammate.next for adaptive and personalized AI partnership. This paper lays the foundation for future discussions on the role of AI in the next era of software engineering.
  arXiv  Detail & Related papers  (2024-10-08T15:04:07Z)
• OpenHands: An Open Platform for AI Software Developers as Generalist Agents [109.8507367518992]
  We introduce OpenHands, a platform for the development of AI agents that interact with the world in similar ways to a human developer.<n>We describe how the platform allows for the implementation of new agents, safe interaction with sandboxed environments for code execution, and incorporation of evaluation benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T17:50:43Z)
• The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
  Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI) We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
  arXiv  Detail & Related papers  (2023-09-14T17:12:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.