How I Learned to Stop Worrying and Love ChatGPT

• URL: http://arxiv.org/abs/2504.05712v1
• Date: Tue, 08 Apr 2025 06:13:16 GMT
• Title: How I Learned to Stop Worrying and Love ChatGPT
• Authors: Piotr Przymus, Mikołaj Fejzer, Jakub Narębski, Krzysztof Stencel,
• Abstract summary: ChatGPT-generated code signifies a distinctive and evolving paradigm in development practices.<n>We aim to provide valuable insights into the transformative role of ChatGPT in software development, illuminating its implications for code evolution and sustainability within the ecosystem.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the dynamic landscape of software engineering, the emergence of ChatGPT-generated code signifies a distinctive and evolving paradigm in development practices. We delve into the impact of interactions with ChatGPT on the software development process, specifically analysing its influence on source code changes. Our emphasis lies in aligning code with ChatGPT conversations, separately analysing the user-provided context of the code and the extent to which the resulting code has been influenced by ChatGPT. Additionally, employing survival analysis techniques, we examine the longevity of ChatGPT-generated code segments in comparison to lines written traditionally. The goal is to provide valuable insights into the transformative role of ChatGPT in software development, illuminating its implications for code evolution and sustainability within the ecosystem.

Related papers

• Distinguishing LLM-generated from Human-written Code by Contrastive Learning [5.553326595990857]
  Large language models (LLMs) have attracted significant attention due to their demonstrated ability to generate high-quality content for various tasks. There are growing concerns regarding their potential risks in various fields, such as news, education, and software engineering. This paper proposes a novel ChatGPT-generated code detector, CodeGPTSensor, based on a contrastive learning framework and a semantic encoder built with UniXcoder.
  arXiv  Detail & Related papers  (2024-11-07T13:39:14Z)
• Impact of the Availability of ChatGPT on Software Development: A Synthetic Difference in Differences Estimation using GitHub Data [49.1574468325115]
  ChatGPT is an AI tool that enhances software production efficiency. We estimate ChatGPT's effects on the number of git pushes, repositories, and unique developers per 100,000 people. These results suggest that AI tools like ChatGPT can substantially boost developer productivity, though further analysis is needed to address potential downsides such as low quality code and privacy concerns.
  arXiv  Detail & Related papers  (2024-06-16T19:11:15Z)
• Investigating the Utility of ChatGPT in the Issue Tracking System: An Exploratory Study [5.176434782905268]
  This study examines the interaction between ChatGPT and developers to analyze their prevalent activities and provide a resolution. Our investigation reveals that developers mainly use ChatGPT for brainstorming solutions but often opt to write their code instead of using ChatGPT-generated code.
  arXiv  Detail & Related papers  (2024-02-06T06:03:05Z)
• Exploring ChatGPT's Capabilities on Vulnerability Management [56.4403395100589]
  We explore ChatGPT's capabilities on 6 tasks involving the complete vulnerability management process with a large-scale dataset containing 70,346 samples. One notable example is ChatGPT's proficiency in tasks like generating titles for software bug reports. Our findings reveal the difficulties encountered by ChatGPT and shed light on promising future directions.
  arXiv  Detail & Related papers  (2023-11-11T11:01:13Z)
• DevGPT: Studying Developer-ChatGPT Conversations [12.69439932665687]
  This paper introduces DevGPT, a dataset curated to explore how software developers interact with ChatGPT. The dataset encompasses 29,778 prompts and responses from ChatGPT, including 19,106 code snippets.
  arXiv  Detail & Related papers  (2023-08-31T06:55:40Z)
• No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT [28.68768157452352]
  This study examines the quality of code generation using ChatGPT. We leverage 728 algorithm problems in five languages (i.e., C, C++, Java, Python, and JavaScript) and 18 CWEs with 54 code scenarios for the code generation task. Our findings uncover potential issues and limitations that arise in the ChatGPT-based code generation.
  arXiv  Detail & Related papers  (2023-08-09T10:01:09Z)
• LMs: Understanding Code Syntax and Semantics for Code Analysis [25.508254718438636]
  We evaluate the capabilities of large language models (LLMs) and their limitations for code analysis in software engineering. We employ four state-of-the-art foundational models, GPT4, GPT3.5, StarCoder and CodeLlama-13b-instruct.
  arXiv  Detail & Related papers  (2023-05-20T08:43:49Z)
• Improving ChatGPT Prompt for Code Generation [13.303599826870705]
  OpenAI's language model ChatGPT has emerged as a powerful tool for generating human-like responses to a wide range of textual inputs. We evaluate ChatGPT's capabilities for two code generation tasks, including text-to-code and code-to-code generation. Our results showed that by carefully designing prompts to guide ChatGPT, the generation performance can be improved substantially.
  arXiv  Detail & Related papers  (2023-05-15T05:37:33Z)
• InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language [82.92236977726655]
  InternGPT stands for textbfinteraction, textbfnonverbal, and textbfchatbots. We present an interactive visual framework named InternGPT, or iGPT for short.
  arXiv  Detail & Related papers  (2023-05-09T17:58:34Z)
• ChatLog: Carefully Evaluating the Evolution of ChatGPT Across Time [54.18651663847874]
  ChatGPT has achieved great success and can be considered to have acquired an infrastructural status. Existing benchmarks encounter two challenges: (1) Disregard for periodical evaluation and (2) Lack of fine-grained features. We construct ChatLog, an ever-updating dataset with large-scale records of diverse long-form ChatGPT responses for 21 NLP benchmarks from March, 2023 to now.
  arXiv  Detail & Related papers  (2023-04-27T11:33:48Z)
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
  Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
  arXiv  Detail & Related papers  (2023-04-12T05:08:52Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Consistency Analysis of ChatGPT [65.268245109828]
  This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions.
  arXiv  Detail & Related papers  (2023-03-11T01:19:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.