Multi-Label Requirements Classification with Large Taxonomies

• URL: http://arxiv.org/abs/2406.04797v2
• Date: Thu, 13 Jun 2024 06:55:22 GMT
• Title: Multi-Label Requirements Classification with Large Taxonomies
• Authors: Waleed Abdeen, Michael Unterkalmsteiner, Krzysztof Wnuk, Alexandros Chirtoglou, Christoph Schimanski, Heja Goli,
• Abstract summary: Multi-label requirements classification with large labels could aid requirements traceability but is prohibitively costly with supervised training. We associated 129 requirements with 769 labels from ranging between 250 and 1183 classes. The sentence-based classification had a significantly higher recall compared to the word-based classification. The hierarchical classification strategy did not always improve the performance of requirements classification.
• Score: 40.588683959176116
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Classification aids software development activities by organizing requirements in classes for easier access and retrieval. The majority of requirements classification research has, so far, focused on binary or multi-class classification. Multi-label classification with large taxonomies could aid requirements traceability but is prohibitively costly with supervised training. Hence, we investigate zero-short learning to evaluate the feasibility of multi-label requirements classification with large taxonomies. We associated, together with domain experts from the industry, 129 requirements with 769 labels from taxonomies ranging between 250 and 1183 classes. Then, we conducted a controlled experiment to study the impact of the type of classifier, the hierarchy, and the structural characteristics of taxonomies on the classification performance. The results show that: (1) The sentence-based classifier had a significantly higher recall compared to the word-based classifier; however, the precision and F1-score did not improve significantly. (2) The hierarchical classification strategy did not always improve the performance of requirements classification. (3) The total and leaf nodes of the taxonomies have a strong negative correlation with the recall of the hierarchical sentence-based classifier. We investigate the problem of multi-label requirements classification with large taxonomies, illustrate a systematic process to create a ground truth involving industry participants, and provide an analysis of different classification pipelines using zero-shot learning.

Related papers

• Hierarchical Query Classification in E-commerce Search [38.67034103433015]
  E-commerce platforms typically store and structure product information and search data in a hierarchy. Efficiently categorizing user search queries into a similar hierarchical structure is paramount in enhancing user experience on e-commerce platforms as well as news curation and academic research. The inherent complexity of hierarchical query classification is compounded by two primary challenges: (1) the pronounced class imbalance that skews towards dominant categories, and (2) the inherent brevity and ambiguity of search queries that hinder accurate classification.
  arXiv  Detail & Related papers  (2024-03-09T21:55:55Z)
• TaxoKnow: Taxonomy as Prior Knowledge in the Loss Function of Multi-class Classification [1.130757825611188]
  We introduce two methods to integrate the hierarchical taxonomy as an explicit regularizer into the loss function of learning algorithms. By reasoning on a hierarchical taxonomy, a neural network alleviates its output distributions over the classes, allowing conditioning on upper concepts for a minority class.
  arXiv  Detail & Related papers  (2023-05-24T08:08:56Z)
• Anomaly Detection using Ensemble Classification and Evidence Theory [62.997667081978825]
  We present a novel approach for novel detection using ensemble classification and evidence theory. A pool selection strategy is presented to build a solid ensemble classifier. We use uncertainty for the anomaly detection approach.
  arXiv  Detail & Related papers  (2022-12-23T00:50:41Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• Semi-Supervised Learning with Taxonomic Labels [42.02670649470055]
  We propose techniques to incorporate coarse taxonomic labels to train image classifiers in fine-grained domains. On the Semi-iNat dataset consisting of 810 species across three Kingdoms, incorporating Phylum labels improves the Species level classification accuracy by 6%. We propose a technique to select relevant data from a large collection of unlabeled images guided by the hierarchy which improves the robustness.
  arXiv  Detail & Related papers  (2021-11-23T00:50:25Z)
• Inducing a hierarchy for multi-class classification problems [11.58041597483471]
  In applications where categorical labels follow a natural hierarchy, classification methods that exploit the label structure often outperform those that do not. In this paper, we investigate a class of methods that induce a hierarchy that can similarly improve classification performance over flat classifiers. We demonstrate the effectiveness of the class of methods both for discovering a latent hierarchy and for improving accuracy in principled simulation settings and three real data applications.
  arXiv  Detail & Related papers  (2021-02-20T05:40:42Z)
• Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View [82.80085730891126]
  We provide the first modernally precise analysis of linear multiclass classification. Our analysis reveals that the classification accuracy is highly distribution-dependent. The insights gained may pave the way for a precise understanding of other classification algorithms.
  arXiv  Detail & Related papers  (2020-11-16T05:17:29Z)
• Classification with Rejection Based on Cost-sensitive Classification [83.50402803131412]
  We propose a novel method of classification with rejection by ensemble of learning. Experimental results demonstrate the usefulness of our proposed approach in clean, noisy, and positive-unlabeled classification.
  arXiv  Detail & Related papers  (2020-10-22T14:05:05Z)
• Cooperative Bi-path Metric for Few-shot Learning [50.98891758059389]
  We make two contributions to investigate the few-shot classification problem. We report a simple and effective baseline trained on base classes in the way of traditional supervised learning. We propose a cooperative bi-path metric for classification, which leverages the correlations between base classes and novel classes to further improve the accuracy.
  arXiv  Detail & Related papers  (2020-08-10T11:28:52Z)
• Joint Embedding of Words and Category Labels for Hierarchical Multi-label Text Classification [4.2750700546937335]
  hierarchical text classification (HTC) has received extensive attention and has broad application prospects. We propose a joint embedding of text and parent category based on hierarchical fine-tuning ordered neurons LSTM (HFT-ONLSTM) for HTC.
  arXiv  Detail & Related papers  (2020-04-06T11:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.