LabelMate: An LLM-Driven Framework for Refined Issue Report Labeling

• URL: http://arxiv.org/abs/2609.04055v1
• Date: Thu, 03 Sep 2026 16:31:42 GMT
• Title: LabelMate: An LLM-Driven Framework for Refined Issue Report Labeling
• Abstract summary: Software users often submit issue reports to a product's issue tracking system to report defects, suggest enhancements, or raise other product-related concerns.<n>Many issue reports remain unlabeled due to the substantial manual effort required to design an appropriate label taxonomy.<n>We propose LabelMate, a novel framework that derives a comprehensive, project-specific label set from historical issue reports.
• Score: 0.3347876022070875
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Software users often submit issue reports to a product's issue tracking system to report defects, suggest enhancements, or raise other product-related concerns. Labeling these issue reports supports effective planning and improves community engagement. However, many issue reports remain unlabeled due to the substantial manual effort required to design an appropriate label taxonomy, then assign suitable labels from this taxonomy to new issue reports. Existing automated labeling approaches attempt to mitigate these challenges. However, they suffer from key limitations, such as extensive manual intervention, the assignment of generic labels, and a dependence on existing labeled datasets. To address these limitations, we propose LabelMate, a novel Large Language Model (LLM)-driven framework that (1) derives a comprehensive, project-specific label set from historical issue reports and (2) automatically assigns relevant labels to new issue reports without requiring any pre-labeled training data. We evaluate LabelMate on 16,500 issue reports from 30 popular and diverse GitHub repositories. Based on this dataset, our approach generates a coherent list of 275 labels and achieves an average labeling accuracy of 89.84%, a statistically significant improvement over existing generic label assigning approaches. These results demonstrate that LabelMate offers an efficient, domain-adaptive solution to streamline the issue labeling process.

Related papers

• Feature-Label Modal Alignment for Robust Partial Multi-Label Learning [66.1611192892514]
  We propose a novel PML method based on feature-label modal alignment (PML-MA)<n>PML-MA treats features and labels as two complementary modalities and restores their consistency through systematic alignment.<n>Experiments on both real-world and synthetic datasets demonstrate that PML-MA significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-10T07:44:46Z)
• Automatic Labelling with Open-source LLMs using Dynamic Label Schema Integration [0.0]
  We explore effectively leveraging open-source models for automatic labelling.<n>We propose Retrieval Augmented Classification (RAC) for which LLM performs inferences for one label at a time.<n>We show that our method, which dynamically integrates label description, leads to performance improvements in labelling tasks.
  arXiv  Detail & Related papers  (2025-01-21T18:06:54Z)
• Label Errors in the Tobacco3482 Dataset [5.9848836847249185]
  Tobacco2 is a widely used document classification benchmark dataset.<n>We find that 11.7% of the dataset is improperly annotated and should either have an unknown label or a corrected label.<n>We then analyze the mistakes of a top-performing model and find that 35% of the model's mistakes can be directly attributed to these label issues.
  arXiv  Detail & Related papers  (2024-12-17T18:06:28Z)
• Substituting Data Annotation with Balanced Updates and Collective Loss in Multi-label Text Classification [19.592985329023733]
  Multi-label text classification (MLTC) is the task of assigning multiple labels to a given text. We study the MLTC problem in annotation-free and scarce-annotation settings in which the magnitude of available supervision signals is linear to the number of labels. Our method follows three steps, (1) mapping input text into a set of preliminary label likelihoods by natural language inference using a pre-trained language model, (2) calculating a signed label dependency graph by label descriptions, and (3) updating the preliminary label likelihoods with message passing along the label dependency graph.
  arXiv  Detail & Related papers  (2023-09-24T04:12:52Z)
• Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations [91.67511167969934]
  imprecise label learning (ILL) is a framework for the unification of learning with various imprecise label configurations. We demonstrate that ILL can seamlessly adapt to partial label learning, semi-supervised learning, noisy label learning, and, more importantly, a mixture of these settings.
  arXiv  Detail & Related papers  (2023-05-22T04:50:28Z)
• Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification [85.76130799062379]
  We study how false negative labels affect the model's explanation. We propose to boost the attribution scores of the model trained with partial labels to make its explanation resemble that of the model trained with full labels.
  arXiv  Detail & Related papers  (2023-04-04T14:00:59Z)
• Label Matching Semi-Supervised Object Detection [85.99282969977541]
  Semi-supervised object detection has made significant progress with the development of mean teacher driven self-training. Label mismatch problem is not yet fully explored in the previous works, leading to severe confirmation bias during self-training. We propose a simple yet effective LabelMatch framework from two different yet complementary perspectives.
  arXiv  Detail & Related papers  (2022-06-14T05:59:41Z)
• Automatic Issue Classifier: A Transfer Learning Framework for Classifying Issue Reports [0.0]
  We use an off-the-shelf neural network called RoBERTa and finetune it to classify the issue reports. This paper presents our approach to classify the issue reports in a multi-label setting. We use an off-the-shelf neural network called RoBERTa and finetune it to classify the issue reports.
  arXiv  Detail & Related papers  (2022-02-12T21:43:08Z)
• Can I Solve It? Identifying APIs Required to Complete OSS Task [16.13269535068818]
  We investigate the feasibility and relevance of labeling issues with the domain of the APIs required to complete the tasks. We leverage the issues' description and the project history to build prediction models, which resulted in precision up to 82% and recall up to 97.8%. Our results can inspire the creation of tools to automatically label issues, helping developers to find tasks that better match their skills.
  arXiv  Detail & Related papers  (2021-03-23T16:16:09Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)
• Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network [61.94394163309688]
  We propose a Label-enhanced Task-Adaptive Projection Network (L-TapNet) based on the state-of-the-art few-shot classification model -- TapNet. Experimental results show that our model significantly outperforms the strongest few-shot learning baseline by 14.64 F1 scores in the one-shot setting.
  arXiv  Detail & Related papers  (2020-06-10T07:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.