Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity

• URL: http://arxiv.org/abs/2601.18345v1
• Date: Mon, 26 Jan 2026 10:34:29 GMT
• Title: Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity
• Authors: Romain Robes Théo Matricon, Thomas Degueule, Andre Hora, Stefano Zacchiroli,
• Abstract summary: coding agents leverage Large Language Models (LLMs) in ways that are markedly different from LLM-based code completion.<n>Unlike LLM-based completion, coding agents leave visible traces in software repositories, enabling the use of MSR techniques to study their impact on SE practices.<n>This paper documents the promises, perils, and perils that we have gathered from studying coding agent activity on GitHub.
• Score: 3.5727010297258732
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In 2025, coding agents have seen a very rapid adoption. Coding agents leverage Large Language Models (LLMs) in ways that are markedly different from LLM-based code completion, making their study critical. Moreover, unlike LLM-based completion, coding agents leave visible traces in software repositories, enabling the use of MSR techniques to study their impact on SE practices. This paper documents the promises, perils, and heuristics that we have gathered from studying coding agent activity on GitHub.

Related papers

• Agentic Much? Adoption of Coding Agents on GitHub [6.395990525268647]
  We present the first large-scale study of the adoption of coding agents on GitHub.<n>We find an estimated adoption rate of 15.85%--22.60%, which is very high for a technology only a few months old--and increasing.<n>At the commit level, we find that commits assisted by coding agents are larger than commits only authored by human developers.
  arXiv  Detail & Related papers  (2026-01-26T10:28:10Z)
• Evaluating and Achieving Controllable Code Completion in Code LLM [89.64782747840225]
  We present the first instruction-guided code completion benchmark, Controllable Code Completion Benchmark (C3-Bench)<n>We reveal substantial gaps in instruction-following capabilities between open-source and advanced proprietary models during code completion tasks.<n>The resulting model, Qwen2.5-Coder-C3, achieves state-of-the-art performance on C3-Bench.
  arXiv  Detail & Related papers  (2026-01-22T11:40:04Z)
• From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence [150.3696990310269]
  Large language models (LLMs) have transformed automated software development by enabling direct translation of natural language descriptions into functional code.<n>We provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs.<n>We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder)
  arXiv  Detail & Related papers  (2025-11-23T17:09:34Z)
• On LLM-Assisted Generation of Smart Contracts from Business Processes [0.08192907805418582]
  Large language models (LLMs) have changed the reality of how software is produced.<n>We present an exploratory study to investigate the use of LLMs for generating smart contract code from business process descriptions.<n>Our results show that LLM performance falls short of the perfect reliability required for smart contract development.
  arXiv  Detail & Related papers  (2025-07-30T20:39:45Z)
• ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding [60.37988508851391]
  Language models (LMs) have become a staple of the code-writing toolbox.<n>Research exploring modifications to Code-LMs' pre-training objectives, geared towards improving data efficiency and better disentangling between syntax and semantics, has been noticeably sparse.<n>In this work, we examine grounding on obfuscated code as a means of helping Code-LMs look beyond the surface-form syntax and enhance their pre-training sample efficiency.
  arXiv  Detail & Related papers  (2025-03-27T23:08:53Z)
• ExeCoder: Empowering Large Language Models with Executability Representation for Code Translation [57.604506522287814]
  Existing large language models (LLMs) only learn the contextual semantics of code during pre-training.<n>We propose ExeCoder to utilize executability representations such as functional semantics, syntax structures, and variable dependencies.<n>We show that ExeCoder achieves state-of-the-art performance in code translation, surpassing existing open-source code LLMs by over 10.88% to 38.78% and over 27.44% to 42.97% on two metrics.
  arXiv  Detail & Related papers  (2025-01-30T16:18:52Z)
• A Survey on Large Language Models for Code Generation [9.555952109820392]
  Large Language Models (LLMs) have garnered remarkable advancements across diverse code-related tasks. This survey aims to bridge the gap between academia and practical development by providing a comprehensive and up-to-date literature review.
  arXiv  Detail & Related papers  (2024-06-01T17:48:15Z)
• Automated Commit Message Generation with Large Language Models: An Empirical Study and Beyond [24.151927600694066]
  Commit Message Generation (CMG) approaches aim to automatically generate commit messages based on given code diffs. This paper conducts the first comprehensive experiment to investigate how far we have been in applying Large Language Models (LLMs) to generate high-quality commit messages.
  arXiv  Detail & Related papers  (2024-04-23T08:24:43Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• Experiential Co-Learning of Software-Developing Agents [83.34027623428096]
  Large language models (LLMs) have brought significant changes to various domains, especially in software development. We introduce Experiential Co-Learning, a novel LLM-agent learning framework. Experiments demonstrate that the framework enables agents to tackle unseen software-developing tasks more effectively.
  arXiv  Detail & Related papers  (2023-12-28T13:50:42Z)
• A Review of Repository Level Prompting for LLMs [0.0]
  Large Language Models (LLMs) have led to notable successes, such as achieving a 94.6% solve rate on the HumanEval benchmark. There is an increasing commercial push for repository-level inline code completion tools, such as GitHub Copilot and Tab Nine. This paper delves into the transition from individual coding problems to repository-scale solutions.
  arXiv  Detail & Related papers  (2023-12-15T00:34:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.