ChatGPT for Code Refactoring: Analyzing Topics, Interaction, and Effective Prompts

• URL: http://arxiv.org/abs/2509.08090v1
• Date: Tue, 09 Sep 2025 18:55:03 GMT
• Title: ChatGPT for Code Refactoring: Analyzing Topics, Interaction, and Effective Prompts
• Authors: Eman Abdullah AlOmar, Luo Xu, Sofia Martinez, Anthony Peruma, Mohamed Wiem Mkaouer, Christian D. Newman, Ali Ouni,
• Abstract summary: Large Language Models (LLMs) have become widely used in various software engineering tasks such as testing, code review, and program comprehension.<n>Our goal is to explore interactions related to between developers and ChatGPT to better understand how developers identify areas for improvement in code.<n>Our approach involves text mining 715-related interactions from 29,778 ChatGPT prompts and responses, as well as the analysis of developers' explicit intentions.
• Score: 10.31253409274086
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Large Language Models (LLMs), such as ChatGPT, have become widely popular and widely used in various software engineering tasks such as refactoring, testing, code review, and program comprehension. Although recent studies have examined the effectiveness of LLMs in recommending and suggesting refactoring, there is a limited understanding of how developers express their refactoring needs when interacting with ChatGPT. In this paper, our goal is to explore interactions related to refactoring between developers and ChatGPT to better understand how developers identify areas for improvement in code, and how ChatGPT addresses developers' needs. Our approach involves text mining 715 refactoring-related interactions from 29,778 ChatGPT prompts and responses, as well as the analysis of developers' explicit refactoring intentions.

Related papers

• Automated Refactoring of Non-Idiomatic Python Code: A Differentiated Replication with LLMs [54.309127753635366]
  We present the results of a replication study in which we investigate GPT-4 effectiveness in recommending and suggesting idiomatic actions.<n>Our findings underscore the potential of LLMs to achieve tasks where, in the past, implementing recommenders based on complex code analyses was required.
  arXiv  Detail & Related papers  (2025-01-28T15:41:54Z)
• How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations [14.736597134188065]
  Little is known about how developers articulate their needs when interacting with ChatGPT. Our approach relies on text mining-related conversations from 17,913 ChatGPT prompts and responses. We envision that our findings contribute to a broader understanding of the collaboration between developers and AI models, in the context of code, with implications for model improvement, tool development, and best practices in software engineering.
  arXiv  Detail & Related papers  (2024-02-08T19:24:01Z)
• Rocks Coding, Not Development--A Human-Centric, Experimental Evaluation of LLM-Supported SE Tasks [9.455579863269714]
  We examined whether and to what degree working with ChatGPT was helpful in the coding task and typical software development task. We found that while ChatGPT performed well in solving simple coding problems, its performance in supporting typical software development tasks was not that good. Our study thus provides first-hand insights into using ChatGPT to fulfill software engineering tasks with real-world developers.
  arXiv  Detail & Related papers  (2024-02-08T13:07:31Z)
• Investigating the Utility of ChatGPT in the Issue Tracking System: An Exploratory Study [5.176434782905268]
  This study examines the interaction between ChatGPT and developers to analyze their prevalent activities and provide a resolution. Our investigation reveals that developers mainly use ChatGPT for brainstorming solutions but often opt to write their code instead of using ChatGPT-generated code.
  arXiv  Detail & Related papers  (2024-02-06T06:03:05Z)
• DevGPT: Studying Developer-ChatGPT Conversations [12.69439932665687]
  This paper introduces DevGPT, a dataset curated to explore how software developers interact with ChatGPT. The dataset encompasses 29,778 prompts and responses from ChatGPT, including 19,106 code snippets.
  arXiv  Detail & Related papers  (2023-08-31T06:55:40Z)
• ChatDev: Communicative Agents for Software Development [84.90400377131962]
  ChatDev is a chat-powered software development framework in which specialized agents are guided in what to communicate. These agents actively contribute to the design, coding, and testing phases through unified language-based communication.
  arXiv  Detail & Related papers  (2023-07-16T02:11:34Z)
• State of Refactoring Adoption: Better Understanding Developer Perception of Refactoring [5.516979718589074]
  We aim to explore how developers document their activities during the software life cycle. We call such activity Self-Affirmed Refactoring (SAR), which indicates developers' documentation of their activities. We propose an approach to identify whether a commit describes developer-related events to classify them according to the common quality improvement categories.
  arXiv  Detail & Related papers  (2023-06-09T16:38:20Z)
• ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models [125.7209927536255]
  We propose ChatCoT, a tool-augmented chain-of-thought reasoning framework for chat-based LLMs. In ChatCoT, we model the chain-of-thought (CoT) reasoning as multi-turn conversations, to utilize tools in a more natural way through chatting. Our approach can effectively leverage the multi-turn conversation ability of chat-based LLMs, and integrate the thought chain following and tools manipulation in a unified way.
  arXiv  Detail & Related papers  (2023-05-23T17:54:33Z)
• Do code refactorings influence the merge effort? [80.1936417993664]
  Multiple contributors frequently change the source code in parallel to implement new features, fix bugs, existing code, and make other changes. These simultaneous changes need to be merged into the same version of the source code. Studies show that 10 to 20 percent of all merge attempts result in conflicts, which require the manual developer's intervention to complete the process.
  arXiv  Detail & Related papers  (2023-05-10T13:24:59Z)
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
  Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
  arXiv  Detail & Related papers  (2023-04-12T05:08:52Z)
• Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension [49.92173751203827]
  In multi-turn dialog, utterances do not always take the full form of sentences. We propose to improve the response generation performance by examining the model's ability to answer a reading comprehension question.
  arXiv  Detail & Related papers  (2020-12-14T10:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.