(Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding

• URL: http://arxiv.org/abs/2607.26375v1
• Date: Wed, 29 Jul 2026 01:23:04 GMT
• Title: (Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding
• Abstract summary: We have 54 students create a website with one of two AI systems: an agent that edits user code.<n>We test understanding via comprehension questions and a task where users extend their code without agents.<n>Despite self-reported weaker understanding, users still prefer coding agents because they are quick and easy to use.
• Score: 59.19634073105828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Coding agents (e.g., Cursor) improve developer productivity by optimizing task completion, but shifting users from writing code to prompting and reviewing may harm their understanding, impeding oversight, learning, and communication. To probe this, we have 54 students create a website with one of two AI systems: an agent that edits user code; or a chatbot where users write code alone or adapt generic code snippets. We test understanding via comprehension questions and a task where users extend their code without agents, showing: (1) While agents aid initial task completion, they harm users' code comprehension and thus do not prepare users to extend their code; (2) Low-effort agent interaction types, like copy+paste prompts and auto-accepted edits, are linked with lower comprehension; and (3) Despite self-reported weaker understanding, users still prefer coding agents because they are quick and easy to use. While users stay in the loop for coding workflows, understanding should not be forgotten. Towards this goal, we distill our analyses into future research directions for coding agent developers: dissuading low-effort prompting, creating readable code, and promoting active engagement.

Related papers

• Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors [52.619793817715326]
  We present a multi-stage crowdsourced study evaluating skill-targeted, LLM-generated Python error messages.<n>We show that while LLM-rewritten messages significantly improved subjective evaluations, these perceived gains did not translate into statistically significant improvements in objective debug performance.<n>This highlights a critical human-AI complementarity gap: explanations that feel better to users do not necessarily make them more effective debuggers.
  arXiv  Detail & Related papers  (2026-08-21T09:14:28Z)
• TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization [57.14479960309637]
  Coding agents are increasingly used to accelerate code generation in many downstream tasks.<n>Despite their value, agent-generated code tends to be larger and more verbose than the corresponding human-written implementation.<n>We show that the cause lies in the agent's own search process.
  arXiv  Detail & Related papers  (2026-07-20T17:06:19Z)
• SWE-chat: Coding Agent Interactions From Real Users in the Wild [70.18158706281724]
  SWE-chat is the first large-scale dataset of real coding agent sessions collected from open-source developers in the wild.<n>The dataset currently contains 6,000 sessions, comprising more than 63,000 user prompts and 355,000 agent tool calls.
  arXiv  Detail & Related papers  (2026-04-22T17:08:19Z)
• Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification [10.964060011243234]
  This paper presents a study of using large language models (LLMs) in modifying existing code.<n>"prompting" serves as the primary interface for developers to communicate intents to LLMs.<n>This study investigates two prompting strategies for LLM-assisted code modification.
  arXiv  Detail & Related papers  (2025-08-02T23:52:49Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [60.04362496037186]
  We present the first controlled study of developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants.<n>Our results show agents can assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• Beyond Autocomplete: Designing CopilotLens Towards Transparent and Explainable AI Coding Agents [4.960232980231203]
  CopilotLens is an interactive framework that reframes code completion from a simple suggestion into a transparent, explainable interaction.<n>CopilotLens operates as an explanation layer that reconstructs the AI agent's "thought process" through a dynamic, two-level interface.
  arXiv  Detail & Related papers  (2025-06-24T23:50:03Z)
• Prompting LLMs for Code Editing: Struggles and Remedies [39.02507244469977]
  Large Language Models (LLMs) are rapidly transforming software engineering, with coding assistants embedded in an IDE becoming increasingly prevalent.<n>This paper addresses part of this gap through a multi-phased investigation of developer interactions with an LLM-powered code editing and transformation feature, Transform Code, in an IDE widely used at Google.<n>We analyze telemetry logs of the feature usage, revealing that frequent re-prompting can be an indicator of developer struggles with using Transform Code.<n>We propose and evaluate a tool, AutoPrompter, for automatically improving prompts by inferring missing information from the surrounding code context, leading to a 27% improvement in
  arXiv  Detail & Related papers  (2025-04-28T18:59:28Z)
• Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents [110.25679611755962]
  Current language model-driven agents often lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. We introduce Intention-in-Interaction (IN3), a novel benchmark designed to inspect users' implicit intentions through explicit queries. We empirically train Mistral-Interact, a powerful model that proactively assesses task vagueness, inquires user intentions, and refines them into actionable goals.
  arXiv  Detail & Related papers  (2024-02-14T14:36:30Z)
• CodeAgent: Autonomous Communicative Agents for Code Review [12.163258651539236]
  This work introduces tool, a novel multi-agent Large Language Model (LLM) system for code review automation. CodeAgent incorporates a supervisory agent, QA-Checker, to ensure that all the agents' contributions address the initial review question. Results demonstrate CodeAgent's effectiveness, contributing to a new state-of-the-art in code review automation.
  arXiv  Detail & Related papers  (2024-02-03T14:43:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.