Practical Guidelines for the Selection and Evaluation of Natural Language Processing Techniques in Requirements Engineering

• URL: http://arxiv.org/abs/2401.01508v3
• Date: Tue, 16 Jul 2024 14:06:02 GMT
• Title: Practical Guidelines for the Selection and Evaluation of Natural Language Processing Techniques in Requirements Engineering
• Authors: Mehrdad Sabetzadeh, Chetan Arora,
• Abstract summary: Natural language (NL) is now a cornerstone of requirements automation. With so many different NLP solution strategies available, it can be challenging to choose the right strategy for a specific RE task. In particular, we discuss how to choose among different strategies such as traditional NLP, feature-based machine learning, and language-model-based methods.
• Score: 8.779031107963942
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural Language Processing (NLP) is now a cornerstone of requirements automation. One compelling factor behind the growing adoption of NLP in Requirements Engineering (RE) is the prevalent use of natural language (NL) for specifying requirements in industry. NLP techniques are commonly used for automatically classifying requirements, extracting important information, e.g., domain models and glossary terms, and performing quality assurance tasks, such as ambiguity handling and completeness checking. With so many different NLP solution strategies available and the possibility of applying machine learning alongside, it can be challenging to choose the right strategy for a specific RE task and to evaluate the resulting solution in an empirically rigorous manner. In this chapter, we present guidelines for the selection of NLP techniques as well as for their evaluation in the context of RE. In particular, we discuss how to choose among different strategies such as traditional NLP, feature-based machine learning, and language-model-based methods. Our ultimate hope for this chapter is to serve as a stepping stone, assisting newcomers to NLP4RE in quickly initiating themselves into the NLP technologies most pertinent to the RE field.

Related papers

• Natural Language Processing for Requirements Traceability [47.93107382627423]
  Traceability plays a crucial role in requirements and software engineering, particularly for safety-critical systems. Natural language processing (NLP) and related techniques have made considerable progress in the past decade.
  arXiv  Detail & Related papers  (2024-05-17T15:17:00Z)
• Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks [3.7236121078812485]
  We investigate the use of Natural Language Inference (NLI) in automating requirements engineering tasks. In particular, we focus on three tasks: requirements classification, identification of requirements specification defects, and detection of conflicts in stakeholders' requirements.
  arXiv  Detail & Related papers  (2024-04-24T20:26:48Z)
• Combatting Human Trafficking in the Cyberspace: A Natural Language Processing-Based Methodology to Analyze the Language in Online Advertisements [55.2480439325792]
  This project tackles the pressing issue of human trafficking in online C2C marketplaces through advanced Natural Language Processing (NLP) techniques. We introduce a novel methodology for generating pseudo-labeled datasets with minimal supervision, serving as a rich resource for training state-of-the-art NLP models. A key contribution is the implementation of an interpretability framework using Integrated Gradients, providing explainable insights crucial for law enforcement.
  arXiv  Detail & Related papers  (2023-11-22T02:45:01Z)
• Natural Language Processing for Requirements Formalization: How to Derive New Approaches? [0.32885740436059047]
  We present and discuss principal ideas and state-of-the-art methodologies from the field of NLP. We discuss two different approaches in detail and highlight the iterative development of rule sets. The presented methods are demonstrated on two industrial use cases from the automotive and railway domains.
  arXiv  Detail & Related papers  (2023-09-23T05:45:19Z)
• Situated Natural Language Explanations [54.083715161895036]
  Natural language explanations (NLEs) are among the most accessible tools for explaining decisions to humans. Existing NLE research perspectives do not take the audience into account. Situated NLE provides a perspective and facilitates further research on the generation and evaluation of explanations.
  arXiv  Detail & Related papers  (2023-08-27T14:14:28Z)
• Requirement Formalisation using Natural Language Processing and Machine Learning: A Systematic Review [11.292853646607888]
  We conducted a systematic literature review to outline the current state-of-the-art of NLP and ML techniques in Requirement Engineering. We found that NLP approaches are the most common NLP techniques used for automatic RF, primary operating on structured and semi-structured data. This study also revealed that Deep Learning (DL) technique are not widely used, instead classical ML techniques are predominant in the surveyed studies.
  arXiv  Detail & Related papers  (2023-03-18T17:36:21Z)
• A Survey of Knowledge Enhanced Pre-trained Language Models [78.56931125512295]
  We present a comprehensive review of Knowledge Enhanced Pre-trained Language Models (KE-PLMs) For NLU, we divide the types of knowledge into four categories: linguistic knowledge, text knowledge, knowledge graph (KG) and rule knowledge. The KE-PLMs for NLG are categorized into KG-based and retrieval-based methods.
  arXiv  Detail & Related papers  (2022-11-11T04:29:02Z)
• The Use of NLP-Based Text Representation Techniques to Support Requirement Engineering Tasks: A Systematic Mapping Review [1.5469452301122177]
  The research direction has changed from the use of lexical and syntactic features to the use of advanced embedding techniques. We identify four gaps in the existing literature, why they matter, and how future research can begin to address them.
  arXiv  Detail & Related papers  (2022-05-17T02:47:26Z)
• Meta Learning for Natural Language Processing: A Survey [88.58260839196019]
  Deep learning has been the mainstream technique in natural language processing (NLP) area. Deep learning requires many labeled data and is less generalizable across domains. Meta-learning is an arising field in machine learning studying approaches to learn better algorithms.
  arXiv  Detail & Related papers  (2022-05-03T13:58:38Z)
• Classification of Natural Language Processing Techniques for Requirements Engineering [6.099346764207287]
  We present our effort to synthesize and organize 57 most frequently used NLP techniques in requirements engineering. We classify these NLP techniques in two ways: first, by their NLP tasks in typical pipelines and second, by their linguist analysis levels.
  arXiv  Detail & Related papers  (2022-04-08T20:28:00Z)
• AdaPrompt: Adaptive Model Training for Prompt-based NLP [77.12071707955889]
  We propose AdaPrompt, adaptively retrieving external data for continual pretraining of PLMs. Experimental results on five NLP benchmarks show that AdaPrompt can improve over standard PLMs in few-shot settings. In zero-shot settings, our method outperforms standard prompt-based methods by up to 26.35% relative error reduction.
  arXiv  Detail & Related papers  (2022-02-10T04:04:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.