The Current Challenges of Software Engineering in the Era of Large Language Models

• URL: http://arxiv.org/abs/2412.14554v2
• Date: Fri, 27 Dec 2024 10:17:12 GMT
• Title: The Current Challenges of Software Engineering in the Era of Large Language Models
• Authors: Cuiyun Gao, Xing Hu, Shan Gao, Xin Xia, Zhi Jin,
• Abstract summary: The paper aims at revisiting the software development life cycle (SDLC) under large language models (LLMs) We achieve 26 key challenges from seven aspects, including software requirement & design, coding assistance, testing code generation, code review, code maintenance, and data, training, and evaluation.
• Score: 29.040350190602567
• License:
• Abstract: With the advent of large language models (LLMs) in the artificial intelligence (AI) area, the field of software engineering (SE) has also witnessed a paradigm shift. These models, by leveraging the power of deep learning and massive amounts of data, have demonstrated an unprecedented capacity to understand, generate, and operate programming languages. They can assist developers in completing a broad spectrum of software development activities, encompassing software design, automated programming, and maintenance, which potentially reduces huge human efforts. Integrating LLMs within the SE landscape (LLM4SE) has become a burgeoning trend, necessitating exploring this emergent landscape's challenges and opportunities. The paper aims at revisiting the software development life cycle (SDLC) under LLMs, and highlighting challenges and opportunities of the new paradigm. The paper first summarizes the overall process of LLM4SE, and then elaborates on the current challenges based on a through discussion. The discussion was held among more than 20 participants from academia and industry, specializing in fields such as software engineering and artificial intelligence. Specifically, we achieve 26 key challenges from seven aspects, including software requirement & design, coding assistance, testing code generation, code review, code maintenance, software vulnerability management, and data, training, and evaluation. We hope the achieved challenges would benefit future research in the LLM4SE field.

Related papers

• LLMs: A Game-Changer for Software Engineers? [0.0]
  Large Language Models (LLMs) like GPT-3 and GPT-4 have emerged as groundbreaking innovations with capabilities that extend far beyond traditional AI applications. Their potential to revolutionize software development has captivated the software engineering (SE) community. This paper argues that LLMs are not just reshaping how software is developed but are redefining the role of developers.
  arXiv  Detail & Related papers  (2024-11-01T17:14:37Z)
• Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement [62.94719119451089]
  Lingma SWE-GPT series learns from and simulating real-world code submission activities. Lingma SWE-GPT 72B resolves 30.20% of GitHub issues, marking a significant improvement in automatic issue resolution.
  arXiv  Detail & Related papers  (2024-11-01T14:27:16Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• Morescient GAI for Software Engineering (Extended Version) [2.4861619769660637]
  We present a vision and roadmap for how "Morescient" GAI models can be engineered, evolved and disseminated according to the principles of open science.
  arXiv  Detail & Related papers  (2024-06-07T07:38:33Z)
• Bridging Gaps, Building Futures: Advancing Software Developer Diversity and Inclusion Through Future-Oriented Research [50.545824691484796]
  We present insights from SE researchers and practitioners on challenges and solutions regarding diversity and inclusion in SE. We share potential utopian and dystopian visions of the future and provide future research directions and implications for academia and industry.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond [84.95530356322621]
  This survey presents a systematic review of the advancements in code intelligence. It covers over 50 representative models and their variants, more than 20 categories of tasks, and an extensive coverage of over 680 related works. Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence.
  arXiv  Detail & Related papers  (2024-03-21T08:54:56Z)
• Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study [72.24266814625685]
  We explore the performance of large language models (LLMs) across the entire software development lifecycle with DevEval. DevEval features four programming languages, multiple domains, high-quality data collection, and carefully designed and verified metrics for each task. Empirical studies show that current LLMs, including GPT-4, fail to solve the challenges presented within DevEval.
  arXiv  Detail & Related papers  (2024-03-13T15:13:44Z)
• Dealing with Data for RE: Mitigating Challenges while using NLP and Generative AI [2.9189409618561966]
  Book chapter explores the evolving landscape of Software Engineering in general, and Requirements Engineering (RE) in particular. We discuss challenges that arise while integrating Natural Language Processing (NLP) and generative AI into enterprise-critical software systems. Book provides practical insights, solutions, and examples to equip readers with the knowledge and tools necessary.
  arXiv  Detail & Related papers  (2024-02-26T19:19:47Z)
• Competition-Level Problems are Effective LLM Evaluators [121.15880285283116]
  This paper aims to evaluate the reasoning capacities of large language models (LLMs) in solving recent programming problems in Codeforces. We first provide a comprehensive evaluation of GPT-4's peiceived zero-shot performance on this task, considering various aspects such as problems' release time, difficulties, and types of errors encountered. Surprisingly, theThoughtived performance of GPT-4 has experienced a cliff like decline in problems after September 2021 consistently across all the difficulties and types of problems.
  arXiv  Detail & Related papers  (2023-12-04T18:58:57Z)
• The Transformative Influence of Large Language Models on Software Development [0.0]
  Generalized Large Language Models (LLMs) have found their way into diverse domains. With LLMs increasingly serving as AI Pair Programming Assistants, it also presents critical challenges and open problems. Preliminary findings underscore pressing concerns about data privacy, bias, and misinformation. We identify 12 open problems that we have identified through our survey, covering these various domains.
  arXiv  Detail & Related papers  (2023-11-28T02:18:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.