Challenges and Practices in Aligning Requirements with Verification and Validation: A Case Study of Six Companies

• URL: http://arxiv.org/abs/2307.12489v1
• Date: Mon, 24 Jul 2023 02:39:53 GMT
• Title: Challenges and Practices in Aligning Requirements with Verification and Validation: A Case Study of Six Companies
• Authors: Elizabeth Bjarnason, Per Runeson, Markus Borg, Michael Unterkalmsteiner, Emelie Engstr\"om, Bj\"orn Regnell, Giedre Sabaliauskaite, Annabella Loconsole, Tony Gorschek, Robert Feldt
• Abstract summary: Weak alignment of requirements engineering with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software developing companies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level.
• Score: 10.508558932045032
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted effort and delays. However, despite the serious implications of weak alignment research and practice both tend to focus on one or the other of RE or VV rather than on the alignment of the two. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software developing companies, involving 10 researchers in a flexible research process for case studies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level. The study identified that human aspects are central, i.e. cooperation and communication, and that requirements engineering practices are a critical basis for alignment. Further, the size of an organisation and its motivation for applying alignment practices, e.g. external enforcement of traceability, are variation factors that play a key role in achieving alignment. Our results provide a strategic roadmap for practitioners improvement work to address alignment challenges. Furthermore, the study provides a foundation for continued research to improve the alignment of RE with VV.

Related papers

• Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks [68.49251303172674]
  State-of-the-art large language models (LLMs) exhibit impressive problem-solving capabilities but may struggle with complex reasoning and factual correctness. Existing methods harness the strengths of chain-of-thought and retrieval-augmented generation (RAG) to decompose a complex problem into simpler steps and apply retrieval to improve factual correctness. We introduce Critic-guided planning with Retrieval-augmentation, CR-Planner, a novel framework that leverages fine-tuned critic models to guide both reasoning and retrieval processes through planning.
  arXiv  Detail & Related papers  (2024-10-02T11:26:02Z)
• Regulatory Requirements Engineering in Large Enterprises: An Interview Study on the European Accessibility Act [16.39543253517196]
  The European Accessibility Act (EAA) impacts the engineering of software products and services. Enterprises conduct Regulatory Impact Analysis (RIA) to consider the effects of regulations on software products offered and formulate requirements at an enterprise level. Despite its practical relevance, we are unaware of any studies on this large-scale regulatory RE process.
  arXiv  Detail & Related papers  (2024-09-11T14:42:35Z)
• Re-TASK: Revisiting LLM Tasks from Capability, Skill, and Knowledge Perspectives [54.14429346914995]
  Chain-of-Thought (CoT) has become a pivotal method for solving complex problems. Large language models (LLMs) often struggle to accurately decompose domain-specific tasks. This paper introduces the Re-TASK framework, a novel theoretical model that revisits LLM tasks from the perspectives of capability, skill, and knowledge.
  arXiv  Detail & Related papers  (2024-08-13T13:58:23Z)
• Designing NLP-based solutions for requirements variability management: experiences from a design science study at Visma [4.063380369801306]
  This experience report outlines the insights gained from applying design science in requirements engineering research in industry. We show and evaluate various strategies to tackle the issue of requirement variability.
  arXiv  Detail & Related papers  (2024-02-11T10:12:01Z)
• Responsible AI Considerations in Text Summarization Research: A Review of Current Practices [89.85174013619883]
  We focus on text summarization, a common NLP task largely overlooked by the responsible AI community. We conduct a multi-round qualitative analysis of 333 summarization papers from the ACL Anthology published between 2020-2022. We focus on how, which, and when responsible AI issues are covered, which relevant stakeholders are considered, and mismatches between stated and realized research goals.
  arXiv  Detail & Related papers  (2023-11-18T15:35:36Z)
• Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey [7.164324501049983]
  Requirements Engineering (RE) can help address many problems when engineering Machine Learning-enabled systems. We conducted a survey to gather practitioner insights into the status quo and problems of RE in ML-enabled systems. We found significant differences in RE practices within ML projects.
  arXiv  Detail & Related papers  (2023-10-10T15:53:50Z)
• Assessing requirements engineering and software test alignment -- Five case studies [8.960336770209476]
  REST-bench is an assessment tool that illustrates the coordination in software development projects and identifies concrete improvement opportunities. We applied REST-bench both in Agile and plan-driven environments, in projects lasting from weeks to years, and staffed as large as 1000 employees.
  arXiv  Detail & Related papers  (2023-08-15T08:43:20Z)
• Challenges in aligning requirements engineering and verification in a large-scale industrial context [7.92131557859946]
  This paper presents preliminary findings of interviews that identify key challenges in aligning requirements and verification processes. The findings of this study can be used by practitioners as a basis for investigating alignment in their organizations.
  arXiv  Detail & Related papers  (2023-07-23T20:08:49Z)
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems [76.79703106646967]
  We develop a value-based assessment framework that visualizes closeness and tensions between values. We give guidelines on how to operationalize them, while opening up the evaluation and deliberation process to a wide range of stakeholders.
  arXiv  Detail & Related papers  (2022-05-09T19:28:32Z)
• An Uncommon Task: Participatory Design in Legal AI [64.54460979588075]
  We examine a notable yet understudied AI design process in the legal domain that took place over a decade ago. We show how an interactive simulation methodology allowed computer scientists and lawyers to become co-designers.
  arXiv  Detail & Related papers  (2022-03-08T15:46:52Z)
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey [53.73359052511171]
  Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. We present a framework for curriculum learning (CL) in RL, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals.
  arXiv  Detail & Related papers  (2020-03-10T20:41:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.