Extreme Multi-label Completion for Semantic Document Labelling with Taxonomy-Aware Parallel Learning

• URL: http://arxiv.org/abs/2412.13809v1
• Date: Wed, 18 Dec 2024 12:57:49 GMT
• Title: Extreme Multi-label Completion for Semantic Document Labelling with Taxonomy-Aware Parallel Learning
• Authors: Julien Audiffren, Christophe Broillet, Ljiljana Dolamic, Philippe Cudré-Mauroux,
• Abstract summary: In Extreme Multi Label Completion (XMLCo), the objective is to predict the missing labels of a collection of documents. We propose a new approach to this problem, TAMLEC, which divides the problem into several taxonomy-Aware Tasks. At inference time, TAMLEC uses the labels available in a document to infer the appropriate tasks and to predict missing labels.
• Score: 13.843237359780069
• License:
• Abstract: In Extreme Multi Label Completion (XMLCo), the objective is to predict the missing labels of a collection of documents. Together with XML Classification, XMLCo is arguably one of the most challenging document classification tasks, as the very high number of labels (at least ten of thousands) is generally very large compared to the number of available labelled documents in the training dataset. Such a task is often accompanied by a taxonomy that encodes the labels organic relationships, and many methods have been proposed to leverage this hierarchy to improve the results of XMLCo algorithms. In this paper, we propose a new approach to this problem, TAMLEC (Taxonomy-Aware Multi-task Learning for Extreme multi-label Completion). TAMLEC divides the problem into several Taxonomy-Aware Tasks, i.e. subsets of labels adapted to the hierarchical paths of the taxonomy, and trains on these tasks using a dynamic Parallel Feature sharing approach, where some parts of the model are shared between tasks while others are task-specific. Then, at inference time, TAMLEC uses the labels available in a document to infer the appropriate tasks and to predict missing labels. To achieve this result, TAMLEC uses a modified transformer architecture that predicts ordered sequences of labels on a Weak-Semilattice structure that is naturally induced by the tasks. This approach yields multiple advantages. First, our experiments on real-world datasets show that TAMLEC outperforms state-of-the-art methods for various XMLCo problems. Second, TAMLEC is by construction particularly suited for few-shots XML tasks, where new tasks or labels are introduced with only few examples, and extensive evaluations highlight its strong performance compared to existing methods.

Related papers

• Exploiting Conjugate Label Information for Multi-Instance Partial-Label Learning [61.00359941983515]
  Multi-instance partial-label learning (MIPL) addresses scenarios where each training sample is represented as a multi-instance bag associated with a candidate label set containing one true label and several false positives. ELIMIPL exploits the conjugate label information to improve the disambiguation performance.
  arXiv  Detail & Related papers  (2024-08-26T15:49:31Z)
• Open-world Multi-label Text Classification with Extremely Weak Supervision [30.85235057480158]
  We study open-world multi-label text classification under extremely weak supervision (XWS) We first utilize the user description to prompt a large language model (LLM) for dominant keyphrases of a subset of raw documents, and then construct a label space via clustering. We then apply a zero-shot multi-label classifier to locate the documents with small top predicted scores, so we can revisit their dominant keyphrases for more long-tail labels. X-MLClass exhibits a remarkable increase in ground-truth label space coverage on various datasets.
  arXiv  Detail & Related papers  (2024-07-08T04:52:49Z)
• Retrieval-augmented Multi-label Text Classification [20.100081284294973]
  Multi-label text classification is a challenging task in settings of large label sets. Retrieval augmentation aims to improve the sample efficiency of classification models. We evaluate this approach on four datasets from the legal and biomedical domains.
  arXiv  Detail & Related papers  (2023-05-22T14:16:23Z)
• Adopting the Multi-answer Questioning Task with an Auxiliary Metric for Extreme Multi-label Text Classification Utilizing the Label Hierarchy [10.87653109398961]
  This paper adopts the multi-answer questioning task for extreme multi-label classification. This study adopts the proposed method and the evaluation metric to the legal domain.
  arXiv  Detail & Related papers  (2023-03-02T08:40:31Z)
• A Survey on Extreme Multi-label Learning [72.8751573611815]
  Multi-label learning has attracted significant attention from both academic and industry field in recent decades. It is infeasible to directly adapt them to extremely large label space because of the compute and memory overhead. eXtreme Multi-label Learning (XML) is becoming an important task and many effective approaches are proposed.
  arXiv  Detail & Related papers  (2022-10-08T08:31:34Z)
• Large Loss Matters in Weakly Supervised Multi-Label Classification [50.262533546999045]
  We first regard unobserved labels as negative labels, casting the W task into noisy multi-label classification. We propose novel methods for W which reject or correct the large loss samples to prevent model from memorizing the noisy label. Our methodology actually works well, validating that treating large loss properly matters in a weakly supervised multi-label classification.
  arXiv  Detail & Related papers  (2022-06-08T08:30:24Z)
• Active Learning in Incomplete Label Multiple Instance Multiple Label Learning [17.5720245903743]
  We propose a novel bag-class pair based approach for active learning in the MIML setting. Our approach is based on a discriminative graphical model with efficient and exact inference.
  arXiv  Detail & Related papers  (2021-07-22T17:01:28Z)
• MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
  MATCH is an end-to-end framework that leverages both metadata and hierarchy information. We propose different ways to regularize the parameters and output probability of each child label by its parents. Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
  arXiv  Detail & Related papers  (2021-02-15T05:23:08Z)
• Adaptive Self-training for Few-shot Neural Sequence Labeling [55.43109437200101]
  We develop techniques to address the label scarcity challenge for neural sequence labeling models. Self-training serves as an effective mechanism to learn from large amounts of unlabeled data. meta-learning helps in adaptive sample re-weighting to mitigate error propagation from noisy pseudo-labels.
  arXiv  Detail & Related papers  (2020-10-07T22:29:05Z)
• Interaction Matching for Long-Tail Multi-Label Classification [57.262792333593644]
  We present an elegant and effective approach for addressing limitations in existing multi-label classification models. By performing soft n-gram interaction matching, we match labels with natural language descriptions.
  arXiv  Detail & Related papers  (2020-05-18T15:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.