From Bugs to Benefits: Improving User Stories by Leveraging Crowd Knowledge with CrUISE-AC

• URL: http://arxiv.org/abs/2501.15181v1
• Date: Sat, 25 Jan 2025 11:44:24 GMT
• Title: From Bugs to Benefits: Improving User Stories by Leveraging Crowd Knowledge with CrUISE-AC
• Authors: Stefan Schwedt, Thomas Ströder,
• Abstract summary: We present CrUISE-AC as a fully automated method that investigates issues and generates non-trivial additional acceptance criteria for a given user story. Our evaluation shows that 80-82% of the generated acceptance criteria add relevant requirements to the user stories.
• Score: 0.0
• License:
• Abstract: Costs for resolving software defects increase exponentially in late stages. Incomplete or ambiguous requirements are one of the biggest sources for defects, since stakeholders might not be able to communicate their needs or fail to share their domain specific knowledge. Combined with insufficient developer experience, teams are prone to constructing incorrect or incomplete features. To prevent this, requirements engineering has to explore knowledge sources beyond stakeholder interviews. Publicly accessible issue trackers for systems within the same application domain hold essential information on identified weaknesses, edge cases, and potential error sources, all documented by actual users. Our research aims at (1) identifying, and (2) leveraging such issues to improve an agile requirements artifact known as a "user story". We present CrUISE-AC (Crowd and User Informed Suggestion Engine for Acceptance Criteria) as a fully automated method that investigates issues and generates non-trivial additional acceptance criteria for a given user story by employing NLP techniques and an ensemble of LLMs. CrUISE- AC was evaluated by five independent experts in two distinct business domains. Our findings suggest that issue trackers hold valuable information pertinent to requirements engineering. Our evaluation shows that 80-82% of the generated acceptance criteria add relevant requirements to the user stories. Limitations are the dependence on accessible input issues and the fact that we do not check generated criteria for being conflict-free or non-overlapping with criteria from other user stories.

Related papers

• Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning [61.17099595835263]
  Gensors is a system that empowers users to define customized sensors supported by the reasoning capabilities of MLLMs. In a user study, participants reported significantly greater sense of control, understanding, and ease of communication when defining sensors using Gensors.
  arXiv  Detail & Related papers  (2025-01-27T01:47:57Z)
• Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation [19.543102037001134]
  Language models (LMs) are known to suffer from hallucinations and misinformation. Retrieval augmented generation (RAG) that retrieves verifiable information from an external knowledge corpus provides a tangible solution to these problems. RAG generation quality is highly dependent on the relevance between a user's query and the retrieved documents.
  arXiv  Detail & Related papers  (2024-10-10T19:14:55Z)
• RAG based Question-Answering for Contextual Response Prediction System [0.4660328753262075]
  Large Language Models (LLMs) have shown versatility in various Natural Language Processing (NLP) tasks. Retrieval Augmented Generation (RAG) emerges as a promising technique to address this challenge. This paper introduces an end-to-end framework that employs LLMs with RAG capabilities for industry use cases.
  arXiv  Detail & Related papers  (2024-09-05T17:14:23Z)
• Establishing Knowledge Preference in Language Models [80.70632813935644]
  Language models are known to encode a great amount of factual knowledge through pretraining. Such knowledge might be insufficient to cater to user requests. When answering questions about ongoing events, the model should use recent news articles to update its response. When some facts are edited in the model, the updated facts should override all prior knowledge learned by the model.
  arXiv  Detail & Related papers  (2024-07-17T23:16:11Z)
• From Chaos to Clarity: Claim Normalization to Empower Fact-Checking [57.024192702939736]
  Claim Normalization (aka ClaimNorm) aims to decompose complex and noisy social media posts into more straightforward and understandable forms. We propose CACN, a pioneering approach that leverages chain-of-thought and claim check-worthiness estimation. Our experiments demonstrate that CACN outperforms several baselines across various evaluation measures.
  arXiv  Detail & Related papers  (2023-10-22T16:07:06Z)
• Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators [78.63553017938911]
  Large language models (LLMs) outperform information retrieval techniques for downstream knowledge-intensive tasks. However, community concerns abound regarding the factuality and potential implications of using this uncensored knowledge. We introduce CONNER, designed to evaluate generated knowledge from six important perspectives.
  arXiv  Detail & Related papers  (2023-10-11T08:22:37Z)
• Mining Reddit Data to Elicit Students' Requirements During COVID-19 Pandemic [2.5475486924467075]
  We propose a shift in requirements elicitation, focusing on gathering feedback related to the problem itself. We conducted a case study on student requirements during the COVID-19 pandemic in a higher education institution. We employed multiple machine-learning and natural language processing techniques to identify requirement sentences.
  arXiv  Detail & Related papers  (2023-07-26T14:26:16Z)
• Algorithmic Fairness Datasets: the Story so Far [68.45921483094705]
  Data-driven algorithms are studied in diverse domains to support critical decisions, directly impacting people's well-being. A growing community of researchers has been investigating the equity of existing algorithms and proposing novel ones, advancing the understanding of risks and opportunities of automated decision-making for historically disadvantaged populations. Progress in fair Machine Learning hinges on data, which can be appropriately used only if adequately documented. Unfortunately, the algorithmic fairness community suffers from a collective data documentation debt caused by a lack of information on specific resources (opacity) and scatteredness of available information (sparsity)
  arXiv  Detail & Related papers  (2022-02-03T17:25:46Z)
• Online Learning Demands in Max-min Fairness [91.37280766977923]
  We describe mechanisms for the allocation of a scarce resource among multiple users in a way that is efficient, fair, and strategy-proof. The mechanism is repeated for multiple rounds and a user's requirements can change on each round. At the end of each round, users provide feedback about the allocation they received, enabling the mechanism to learn user preferences over time.
  arXiv  Detail & Related papers  (2020-12-15T22:15:20Z)
• User Intention Recognition and Requirement Elicitation Method for Conversational AI Services [7.941589241861105]
  We aim to obtain user requirements as accurately as possible in as few rounds as possible. A user intention recognition method based on Knowledge Graph (KG) was developed for fuzzy requirement inference. A requirement elicitation method based on Granular Computing was proposed for dialog policy generation.
  arXiv  Detail & Related papers  (2020-09-03T08:26:39Z)
• Enabling the Analysis of Personality Aspects in Recommender Systems [0.0]
  Existing recommender systems mainly focus on exploiting users' feedback, e.g., ratings, and reviews on common items to detect similar users. We call this problem the Data Sparsity With no Feedback on Common Items (DSW-n-FCI) We identify users' personality type implicitly with no burden on users and incorporate it along with users' personal interests and their level of knowledge.
  arXiv  Detail & Related papers  (2020-01-07T23:02:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.