Code Review is a Conversation: Toward Conversational AI Review Assistants

• URL: http://arxiv.org/abs/2607.22095v1
• Date: Fri, 24 Jul 2026 08:44:05 GMT
• Title: Code Review is a Conversation: Toward Conversational AI Review Assistants
• Abstract summary: We argue for conversational AI review assistants that participate in code review as interactive partners rather than static comment generators.<n>Such assistants should identify when conversation is needed, ask grounded questions, respond to developer explanations, summarize unresolved issues, help capture rationale, and know when to abstain or escalate to human reviewers.
• Score: 7.335667281862373
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI-based code review tools increasingly promise to help developers inspect pull requests, identify defects, and improve code quality. Yet most current approaches frame code review as a one-shot commenting task: given a diff, the system produces warnings or suggestions. This framing overlooks a central property of modern code review: review is a conversation. Human reviewers do not merely comment on code; they ask questions, explain expectations, negotiate design trade-offs, request evidence, transfer project knowledge, document rationale, and collectively decide whether a change is good enough to integrate. In this vision paper, we argue for conversational AI review assistants: systems that participate in code review as interactive partners rather than static comment generators. Such assistants should identify when conversation is needed, ask grounded questions, respond to developer explanations, summarize unresolved issues, help capture rationale, and know when to abstain or escalate to human reviewers. Such a paradigm shift requires novel evaluation methodologies as well. We outline a research agenda for studying review conversations, designing conversational AI review capabilities, and evaluating their impact on software evolution and maintenance. Our vision reframes AI code review from automated commenting to human-AI sensemaking before integration.

Related papers

• (Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding [59.19634073105828]
  We have 54 students create a website with one of two AI systems: an agent that edits user code.<n>We test understanding via comprehension questions and a task where users extend their code without agents.<n>Despite self-reported weaker understanding, users still prefer coding agents because they are quick and easy to use.
  arXiv  Detail & Related papers  (2026-07-29T01:23:04Z)
• Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review [2.2807488730605296]
  We present a vision for an AI-powered code review workflow combining specialized agents with human-controlled quality.<n>We identify major open challenges for responsible adoption, including reliability, bias, privacy, automation bias, transparency, and evaluation.
  arXiv  Detail & Related papers  (2026-05-17T17:04:21Z)
• Stop Automating Peer Review Without Rigorous Evaluation [51.53099943385505]
  This position paper argues that today's AI systems should not be used to produce paper reviews.<n>We identify two critical issues: 1) AI reviewers exhibit a hivemind effect of excessive agreement within and across papers that reduces perspective diversity.<n>We argue that addressing the peer review crisis requires a science of peer review automation.
  arXiv  Detail & Related papers  (2026-05-04T22:41:04Z)
• When AI reviews science: Can we trust the referee? [73.47745294608072]
  We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
  arXiv  Detail & Related papers  (2026-04-26T08:03:32Z)
• Human-AI Synergy in Agentic Code Review [3.7086626614863984]
  We compare the feedback differences provided by human reviewers and AI agents.<n>Human reviewers exchange 11.8% more rounds when reviewing AI-generated code than human-written code.<n>Over half of unadopted suggestions from AI agents are either incorrect or addressed through alternative fixes by developers.
  arXiv  Detail & Related papers  (2026-03-16T20:56:18Z)
• CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection [60.52240468810558]
  We introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews.<n>We also develop CoCoDet, an AI review detector via a multi-task learning framework, to achieve more accurate and robust detection of AI involvement in review content.
  arXiv  Detail & Related papers  (2025-08-28T06:03:11Z)
• Code Review as Decision-Making -- Building a Cognitive Model from the Questions Asked During Code Review [2.8299846354183953]
  We build a cognitive model of code review bottom up through thematic, statistical, temporal, and sequential analysis of the transcribed material.<n>The model shows how developers move through two phases during the code review; first an orientation phase to establish context and rationale, then an analytical phase to understand, assess, and plan the rest of the review.
  arXiv  Detail & Related papers  (2025-07-13T14:04:16Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• Deep Learning-based Code Reviews: A Paradigm Shift or a Double-Edged Sword? [14.970843824847956]
  We run a controlled experiment with 29 experts who reviewed different programs with/without the support of an automatically generated code review.<n>We show that reviewers consider valid most of the issues automatically identified by the LLM and that the availability of an automated review as a starting point strongly influences their behavior.<n>The reviewers who started from an automated review identified a higher number of low-severity issues while, however, not identifying more high-severity issues as compared to a completely manual process.
  arXiv  Detail & Related papers  (2024-11-18T09:24:01Z)
• Suggestion Bot: Analyzing the Impact of Automated Suggested Changes on Code Reviews [2.773900417167691]
  We created a bot called SUGGESTION BOT to automatically review the code base using GitHub's suggested changes functionality. We evaluate SUGGESTION BOT concerning its impact on review time and also analyze whether the comments given by the bot are clear and useful for users.
  arXiv  Detail & Related papers  (2023-05-10T17:33:43Z)
• Can We Automate Scientific Reviewing? [89.50052670307434]
  We discuss the possibility of using state-of-the-art natural language processing (NLP) models to generate first-pass peer reviews for scientific papers. We collect a dataset of papers in the machine learning domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers to generate reviews. Comprehensive experimental results show that system-generated reviews tend to touch upon more aspects of the paper than human-written reviews.
  arXiv  Detail & Related papers  (2021-01-30T07:16:53Z)
• Deep Just-In-Time Inconsistency Detection Between Comments and Source Code [51.00904399653609]
  In this paper, we aim to detect whether a comment becomes inconsistent as a result of changes to the corresponding body of code. We develop a deep-learning approach that learns to correlate a comment with code changes. We show the usefulness of our approach by combining it with a comment update model to build a more comprehensive automatic comment maintenance system.
  arXiv  Detail & Related papers  (2020-10-04T16:49:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.