Qualitative software engineering research -- reflections and guidelines

• URL: http://arxiv.org/abs/1712.08341v3
• Date: Mon, 10 Jul 2023 08:57:07 GMT
• Title: Qualitative software engineering research -- reflections and guidelines
• Authors: Per Lenberg, Robert Feldt, Lucas Gren, Lars G\"oran Wallgren Tengberg, Inga Tidefors, Daniel Graziotin
• Abstract summary: We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research. Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.
• Score: 18.630596256915794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Researchers are increasingly recognizing the importance of human aspects in software development. Since qualitative methods are used to explore human behavior in-depth, we believe that studies using such methods will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and the knowledge on how to use them like in social sciences. The purpose of this study was to extend the software engineering community's current body of knowledge regarding available qualitative methods and their quality assurance frameworks, and to provide recommendations and guidelines for their use. With the support of an epistemological argument and a survey of the literature, we suggest that future research would benefit from (1) utilizing a broader set of research methods, (2) more strongly emphasizing reflexivity, and (3) employing qualitative guidelines and quality criteria. We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research, namely interpretative phenomenological analysis, narrative analysis, and discourse analysis. Furthermore, we discuss the meaning of reflexivity in relation to the software engineering context and suggest means of fostering it. Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.

Related papers

• Towards an Understanding of Context Utilization in Code Intelligence [37.85380387094615]
  Code intelligence aims to improve the effectiveness and efficiency of various code-related tasks. Recent research suggests that incorporating contextual information beyond the basic original task inputs can substantially enhance model performance. Despite growing academic interest, there is a lack of systematic analysis of context in code intelligence.
  arXiv  Detail & Related papers  (2025-04-11T17:59:53Z)
• Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation [58.064940977804596]
  A plethora of new AI models and tools has been proposed, promising to empower researchers and academics worldwide to conduct their research more effectively and efficiently. Ethical concerns regarding shortcomings of these tools and potential for misuse take a particularly prominent place in our discussion.
  arXiv  Detail & Related papers  (2025-02-07T18:26:45Z)
• Applications and Implications of Large Language Models in Qualitative Analysis: A New Frontier for Empirical Software Engineering [0.46426852157920906]
  The study emphasizes the need for structured strategies and guidelines to optimize LLM use in qualitative research within software engineering. While LLMs show promise in supporting qualitative analysis, human expertise remains crucial for interpreting data, and ongoing exploration of best practices will be vital for their successful integration into empirical software engineering research.
  arXiv  Detail & Related papers  (2024-12-09T15:17:36Z)
• Large Language Model for Qualitative Research -- A Systematic Mapping Study [3.302912592091359]
  Large Language Models (LLMs), powered by advanced generative AI, have emerged as transformative tools. This study systematically maps the literature on the use of LLMs for qualitative research. Findings reveal that LLMs are utilized across diverse fields, demonstrating the potential to automate processes.
  arXiv  Detail & Related papers  (2024-11-18T21:28:00Z)
• Model-Guided Fieldwork: A Practical, Methodological and Philosophical Investigation in the use of Ethnomethodology for Engineering Software Requirements [0.0]
  This thesis focuses on the use of ethnomethodological fieldwork for the engineering of software requirements. It proposes an approach, dubbed "Model Guided Fieldwork," to support a fieldworker in making observations that may contribute to a technological development process.
  arXiv  Detail & Related papers  (2024-11-14T09:24:56Z)
• Ontology Embedding: A Survey of Methods, Applications and Resources [54.3453925775069]
  Ontologies are widely used for representing domain knowledge and meta data. One straightforward solution is to integrate statistical analysis and machine learning. Numerous papers have been published on embedding, but a lack of systematic reviews hinders researchers from gaining a comprehensive understanding of this field.
  arXiv  Detail & Related papers  (2024-06-16T14:49:19Z)
• Qualitative Data Analysis in Software Engineering: Techniques and Teaching Insights [10.222207222039048]
  Software repositories are rich sources of qualitative artifacts, including source code comments, commit messages, issue descriptions, and documentation. This chapter shifts the focus towards interpreting these artifacts using various qualitative data analysis techniques. Various coding methods are discussed along with the strategic design of a coding guide to ensure consistency and accuracy in data interpretation.
  arXiv  Detail & Related papers  (2024-06-12T13:56:55Z)
• Guidelines for Using Mixed Methods Research in Software Engineering [51.22583433491887]
  Mixed methods research is often used in software engineering, but researchers outside of the social or human sciences often lack experience when using these designs. This paper provides guidelines and advice on how to design mixed method research, and to encourage the intentional, rigorous, and innovative use of mixed methods in software engineering.
  arXiv  Detail & Related papers  (2024-04-09T04:34:25Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [55.33653554387953]
  Pattern Analysis and Machine Intelligence (PAMI) has led to numerous literature reviews aimed at collecting and fragmented information. This paper presents a thorough analysis of these literature reviews within the PAMI field. We try to address three core research questions: (1) What are the prevalent structural and statistical characteristics of PAMI literature reviews; (2) What strategies can researchers employ to efficiently navigate the growing corpus of reviews; and (3) What are the advantages and limitations of AI-generated reviews compared to human-authored ones.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Interactive Multi-Objective Evolutionary Optimization of Software Architectures [0.0]
  Putting the human in the loop brings new challenges to the search-based software engineering field. This paper explores how the interactive evolutionary computation can serve as a basis for integrating the human's judgment into the search process.
  arXiv  Detail & Related papers  (2024-01-08T19:15:40Z)
• Knowledge-enhanced Neural Machine Reasoning: A Review [67.51157900655207]
  We introduce a novel taxonomy that categorizes existing knowledge-enhanced methods into two primary categories and four subcategories. We elucidate the current application domains and provide insight into promising prospects for future research.
  arXiv  Detail & Related papers  (2023-02-04T04:54:30Z)
• Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods [48.47413103662829]
  Natural Language Generation (NLG) has made great progress in recent years due to the development of deep learning techniques such as pre-trained language models. However, the faithfulness problem that the generated text usually contains unfaithful or non-factual information has become the biggest challenge.
  arXiv  Detail & Related papers  (2022-03-10T08:28:32Z)
• Image Quality Assessment in the Modern Age [53.19271326110551]
  This tutorial provides the audience with the basic theories, methodologies, and current progresses of image quality assessment (IQA) We will first revisit several subjective quality assessment methodologies, with emphasis on how to properly select visual stimuli. Both hand-engineered and (deep) learning-based methods will be covered.
  arXiv  Detail & Related papers  (2021-10-19T02:38:46Z)
• Qualitative Investigation in Explainable Artificial Intelligence: A Bit More Insight from Social Science [0.5801044612920815]
  We present a focused analysis of user studies in explainable artificial intelligence (XAI) We draw on social science corpora to suggest ways for improving the rigor of studies where XAI researchers use observations, interviews, focus groups, and/or questionnaires to capture qualitative data. The results of our analysis support calls from others in the XAI community advocating for collaboration with experts from social disciplines to bolster rigor and effectiveness in user studies.
  arXiv  Detail & Related papers  (2020-11-13T21:02:16Z)
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges [76.20963684020145]
  Uncertainty quantification (UQ) plays a pivotal role in reduction of uncertainties during both optimization and decision making processes. Bizarre approximation and ensemble learning techniques are two most widely-used UQ methods in the literature. This study reviews recent advances in UQ methods used in deep learning and investigates the application of these methods in reinforcement learning.
  arXiv  Detail & Related papers  (2020-11-12T06:41:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.