GenAI Is No Silver Bullet for Qualitative Research in Software Engineering

• URL: http://arxiv.org/abs/2603.08951v1
• Date: Mon, 09 Mar 2026 21:35:36 GMT
• Title: GenAI Is No Silver Bullet for Qualitative Research in Software Engineering
• Abstract summary: We discuss the emerging use of generative AI (GenAI) in relation to the broad spectrum of qualitative research in software engineering.<n>Our goal is to inform researchers about the promises and pitfalls of GenAI-assisted qualitative research.
• Score: 12.444212651043593
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Qualitative research gives rich insights into the quintessentially human aspects of software engineering as a socio-technical system. Qualitative research spans diverse strategies and methods, from interpretivist, in situ observational field studies, to deductive coding of data from mining studies. Advances in large language models and generative AI (GenAI) have prompted claims that artificial intelligence could automate qualitative analysis. Such claims are overgeneralizing from narrow successes. GenAI support must be carefully adapted to the data of interest, but also to the characteristics of a particular research strategy. In this Frontiers of SE paper, we discuss the emerging use of GenAI in relation to the broad spectrum of qualitative research in software engineering. We outline the dimensions of qualitative work in software engineering, review emerging empirical evidence for GenAI assistance, examine the pros and cons of GenAI-mediated qualitative research practices, and revisit qualitative research quality factors, in light of GenAI. Our goal is to inform researchers about the promises and pitfalls of GenAI-assisted qualitative research. We conclude with future plans to advance understanding of its use in software engineering.

Related papers

• AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery [55.70879973230979]
  AutoResearchBench is a benchmark for autonomous scientific literature discovery.<n>Compared to previous benchmarks on agentic web browsing, AutoResearchBench is research-oriented.<n>Even the most powerful LLMs, despite having largely conquered general agentic web-browsing benchmarks such as BrowseComp, achieve only 9.39% accuracy on Deep Research and 9.31% IoU on Wide Research.
  arXiv  Detail & Related papers  (2026-04-28T06:05:17Z)
• Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscape [10.576800319451133]
  We conduct a large-scale survey of software engineering (SE) researchers publishing in top venues between 2023 and 2025.<n>We examine who uses Generative AI (GenAI) and why, where it is used across research activities, and how researchers perceive its benefits, opportunities, challenges, risks, and governance.
  arXiv  Detail & Related papers  (2026-04-13T08:43:56Z)
• Generative Artificial Intelligence in Qualitative Research Methods: Between Hype and Risks? [0.0]
  This position paper critically interrogates the role of generative AI (genAI) in the context of qualitative coding methodologies.<n>Despite widespread hype and claims of efficiency, we propose that genAI is not methodologically valid within qualitative inquiries.
  arXiv  Detail & Related papers  (2025-11-11T17:04:24Z)
• AI4Research: A Survey of Artificial Intelligence for Scientific Research [55.5452803680643]
  We present a comprehensive survey on AI for Research (AI4Research)<n>We first introduce a systematic taxonomy to classify five mainstream tasks in AI4Research.<n>We identify key research gaps and highlight promising future directions.
  arXiv  Detail & Related papers  (2025-07-02T17:19:20Z)
• From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models [44.99833362998488]
  We investigated the first steps for optimizing content creation for advanced math.<n>We looked at the ability of GenAI to produce high-quality practice problems that are relevant to the course content.
  arXiv  Detail & Related papers  (2025-05-17T08:30:10Z)
• Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation [58.064940977804596]
  A plethora of new AI models and tools has been proposed, promising to empower researchers and academics worldwide to conduct their research more effectively and efficiently.<n>Ethical concerns regarding shortcomings of these tools and potential for misuse take a particularly prominent place in our discussion.
  arXiv  Detail & Related papers  (2025-02-07T18:26:45Z)
• Navigating Ethical Challenges in Generative AI-Enhanced Research: The ETHICAL Framework for Responsible Generative AI Use [0.0]
  The rapid adoption of generative artificial intelligence (GenAI) in research presents both opportunities and ethical challenges.<n>This paper develops the ETHICAL framework, which is a practical guide for responsible GenAI use in research.
  arXiv  Detail & Related papers  (2024-12-11T05:49:11Z)
• "So what if I used GenAI?" -- Implications of Using Cloud-based GenAI in Software Engineering Research [0.0]
  This paper sheds light on the various research aspects in which GenAI is used, thus raising awareness of its legal implications to novice and budding researchers.<n>We summarize key aspects regarding our current knowledge that every software researcher involved in using GenAI should be aware of to avoid critical mistakes that may expose them to liability claims.
  arXiv  Detail & Related papers  (2024-12-10T06:18:15Z)
• Generative AI Tools in Academic Research: Applications and Implications for Qualitative and Quantitative Research Methodologies [0.0]
  This study examines the impact of Generative Artificial Intelligence (GenAI) on academic research, focusing on its application to qualitative and quantitative data analysis. GenAI tools evolve rapidly, they offer new possibilities for enhancing research productivity and democratising complex analytical processes. Their integration into academic practice raises significant questions regarding research integrity and security, authorship, and the changing nature of scholarly work.
  arXiv  Detail & Related papers  (2024-08-13T13:10:03Z)
• Model-based Maintenance and Evolution with GenAI: A Look into the Future [47.93555901495955]
  We argue that Generative Artificial Intelligence (GenAI) can be used as a means to address the limitations of Model-Based Engineering (MBM&E) We propose that GenAI can be used in MBM&E for: reducing engineers' learning curve, maximizing efficiency with recommendations, or serving as a reasoning tool to understand domain problems.
  arXiv  Detail & Related papers  (2024-07-09T23:13:26Z)
• Generative Artificial Intelligence for Software Engineering -- A Research Agenda [8.685607624226037]
  We conducted a literature review and focus groups for a duration of five months to develop a research agenda on GenAI for Software Engineering. Our results show that it is possible to explore the adoption of GenAI in partial automation and support decision-making in all software development activities. Common considerations when implementing GenAI include industry-level assessment, dependability and accuracy, data accessibility, transparency, and sustainability aspects associated with the technology.
  arXiv  Detail & Related papers  (2023-10-28T09:14:39Z)
• A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics [46.025337523478825]
  Talent analytics has emerged as a promising field in applied data science for human resource management.<n>Recent development of Big Data and Artificial Intelligence techniques have revolutionized human resource management.
  arXiv  Detail & Related papers  (2023-07-03T07:53:20Z)
• Data-centric Artificial Intelligence: A Survey [47.24049907785989]
  Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle.
  arXiv  Detail & Related papers  (2023-03-17T17:44:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.