From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Paper

• URL: http://arxiv.org/abs/2503.07450v3
• Date: Sun, 20 Apr 2025 22:50:35 GMT
• Title: From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Paper
• Authors: Sargam Yadav, Asifa Mehmood Qureshi, Abhishek Kaushik, Shubham Sharma, Roisin Loughran, Subramaniam Kazhuparambil, Andrew Shaw, Mohammed Sabry, Niamh St John Lynch, . Nikhil Singh, Padraic O'Hara, Pranay Jaiswal, Roshan Chandru, David Lillis,
• Abstract summary: The introduction of transformer architecture was a turning point in Natural Language Processing (NLP)<n>Large Language Models (LLMs) such as ChatGPT and Bard to the general public has showcased the tremendous potential of these models.<n>The overall opinion of the experts is positive, with the experts identifying advantages such as increase in productivity and reduced coding time.
• Score: 1.4237262259590389
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The introduction of transformer architecture was a turning point in Natural Language Processing (NLP). Models based on the transformer architecture such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-Trained Transformer (GPT) have gained widespread popularity in various applications such as software development and education. The availability of Large Language Models (LLMs) such as ChatGPT and Bard to the general public has showcased the tremendous potential of these models and encouraged their integration into various domains such as software development for tasks such as code generation, debugging, and documentation generation. In this study, opinions from 11 experts regarding their experience with LLMs for software development have been gathered and analysed to draw insights that can guide successful and responsible integration. The overall opinion of the experts is positive, with the experts identifying advantages such as increase in productivity and reduced coding time. Potential concerns and challenges such as risk of over-dependence and ethical considerations have also been highlighted.

Related papers

• Large Language Models for Code Generation: The Practitioners Perspective [4.946128083535776]
  Large Language Models (LLMs) have emerged as coding assistants, capable of generating source code from natural language prompts. We propose and develop a multi-model unified platform to generate and execute code based on natural language prompts. We conducted a survey with 60 software practitioners from 11 countries across four continents to evaluate the usability, performance, strengths, and limitations of each model.
  arXiv  Detail & Related papers  (2025-01-28T14:52:16Z)
• Darkit: A User-Friendly Software Toolkit for Spiking Large Language Model [50.37090759139591]
  Large language models (LLMs) have been widely applied in various practical applications, typically comprising billions of parameters. The human brain, employing bio-plausible spiking mechanisms, can accomplish the same tasks while significantly reducing energy consumption. We are releasing a software toolkit named DarwinKit (Darkit) to accelerate the adoption of brain-inspired large language models.
  arXiv  Detail & Related papers  (2024-12-20T07:50:08Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models [54.78329741186446]
  We propose a novel paradigm that uses a code-based critic model to guide steps including question-code data construction, quality control, and complementary evaluation. Experiments across both in-domain and out-of-domain benchmarks in English and Chinese demonstrate the effectiveness of the proposed paradigm.
  arXiv  Detail & Related papers  (2024-08-28T06:33:03Z)
• A Survey on Large Language Models from Concept to Implementation [4.219910716090213]
  Recent advancements in Large Language Models (LLMs) have broadened the scope of natural language processing (NLP) applications. This paper investigates the multifaceted applications of these models, with an emphasis on the GPT series. This exploration focuses on the transformative impact of artificial intelligence (AI) driven tools in revolutionizing traditional tasks like coding and problem-solving.
  arXiv  Detail & Related papers  (2024-03-27T19:35:41Z)
• Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study [72.24266814625685]
  We explore the performance of large language models (LLMs) across the entire software development lifecycle with DevEval. DevEval features four programming languages, multiple domains, high-quality data collection, and carefully designed and verified metrics for each task. Empirical studies show that current LLMs, including GPT-4, fail to solve the challenges presented within DevEval.
  arXiv  Detail & Related papers  (2024-03-13T15:13:44Z)
• Can ChatGPT Support Developers? An Empirical Evaluation of Large Language Models for Code Generation [2.93322471069531]
  We conduct an empirical analysis of conversations in DevGPT, a dataset collected from developers' conversations with ChatGPT. Our findings indicate that the current practice of using LLM-generated code is typically limited to either demonstrating high-level concepts or providing examples in documentation.
  arXiv  Detail & Related papers  (2024-02-18T20:48:09Z)
• Redefining Developer Assistance: Through Large Language Models in Software Ecosystem [0.5580128181112308]
  We introduce DevAssistLlama, a model developed through instruction tuning, to assist developers in processing software-related natural language queries. DevAssistLlama is particularly adept at handling intricate technical documentation, enhancing developer capability in software specific tasks.
  arXiv  Detail & Related papers  (2023-12-09T18:02:37Z)
• The Transformative Influence of Large Language Models on Software Development [0.0]
  Generalized Large Language Models (LLMs) have found their way into diverse domains. With LLMs increasingly serving as AI Pair Programming Assistants, it also presents critical challenges and open problems. Preliminary findings underscore pressing concerns about data privacy, bias, and misinformation. We identify 12 open problems that we have identified through our survey, covering these various domains.
  arXiv  Detail & Related papers  (2023-11-28T02:18:54Z)
• L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models [102.00201523306986]
  We present L2CEval, a systematic evaluation of the language-to-code generation capabilities of large language models (LLMs) We analyze the factors that potentially affect their performance, such as model size, pretraining data, instruction tuning, and different prompting methods. In addition to assessing model performance, we measure confidence calibration for the models and conduct human evaluations of the output programs.
  arXiv  Detail & Related papers  (2023-09-29T17:57:00Z)
• ChatDev: Communicative Agents for Software Development [84.90400377131962]
  ChatDev is a chat-powered software development framework in which specialized agents are guided in what to communicate. These agents actively contribute to the design, coding, and testing phases through unified language-based communication.
  arXiv  Detail & Related papers  (2023-07-16T02:11:34Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.