Retrieval-augmented Multi-label Text Classification

• URL: http://arxiv.org/abs/2305.13058v1
• Date: Mon, 22 May 2023 14:16:23 GMT
• Title: Retrieval-augmented Multi-label Text Classification
• Authors: Ilias Chalkidis and Yova Kementchedjhieva
• Abstract summary: Multi-label text classification is a challenging task in settings of large label sets. Retrieval augmentation aims to improve the sample efficiency of classification models. We evaluate this approach on four datasets from the legal and biomedical domains.
• Score: 20.100081284294973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-label text classification (MLC) is a challenging task in settings of large label sets, where label support follows a Zipfian distribution. In this paper, we address this problem through retrieval augmentation, aiming to improve the sample efficiency of classification models. Our approach closely follows the standard MLC architecture of a Transformer-based encoder paired with a set of classification heads. In our case, however, the input document representation is augmented through cross-attention to similar documents retrieved from the training set and represented in a task-specific manner. We evaluate this approach on four datasets from the legal and biomedical domains, all of which feature highly skewed label distributions. Our experiments show that retrieval augmentation substantially improves model performance on the long tail of infrequent labels especially so for lower-resource training scenarios and more challenging long-document data scenarios.

Related papers

• Improving Large-Scale k-Nearest Neighbor Text Categorization with Label Autoencoders [0.0]
  We introduce a multi-label lazy learning approach to deal with automatic semantic indexing in large document collections. The proposed method is an evolution of the traditional k-Nearest Neighbors algorithm. We have evaluated our proposal in a large portion of the MEDLINE biomedical document collection.
  arXiv  Detail & Related papers  (2024-02-03T00:11:29Z)
• Label Semantic Aware Pre-training for Few-shot Text Classification [53.80908620663974]
  We propose Label Semantic Aware Pre-training (LSAP) to improve the generalization and data efficiency of text classification systems. LSAP incorporates label semantics into pre-trained generative models (T5 in our case) by performing secondary pre-training on labeled sentences from a variety of domains.
  arXiv  Detail & Related papers  (2022-04-14T17:33:34Z)
• Long-tailed Extreme Multi-label Text Classification with Generated Pseudo Label Descriptions [28.416742933744942]
  This paper addresses the challenge of tail label prediction by proposing a novel approach. It combines the effectiveness of a trained bag-of-words (BoW) classifier in generating informative label descriptions under severe data scarce conditions. The proposed approach achieves state-of-the-art performance on XMTC benchmark datasets and significantly outperforms the best methods so far in the tail label prediction.
  arXiv  Detail & Related papers  (2022-04-02T23:42:32Z)
• Exploiting Local and Global Features in Transformer-based Extreme Multi-label Text Classification [28.28186933768281]
  We propose an approach that combines both the local and global features produced by Transformer models to improve the prediction power of the classifier. Our experiments show that the proposed model either outperforms or is comparable to the state-of-the-art methods on benchmark datasets.
  arXiv  Detail & Related papers  (2022-04-02T19:55:23Z)
• Multitask Learning for Class-Imbalanced Discourse Classification [74.41900374452472]
  We show that a multitask approach can improve 7% Micro F1-score upon current state-of-the-art benchmarks. We also offer a comparative review of additional techniques proposed to address resource-poor problems in NLP.
  arXiv  Detail & Related papers  (2021-01-02T07:13:41Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)
• Learning Image Labels On-the-fly for Training Robust Classification Models [13.669654965671604]
  We show how noisy annotations (e.g., from different algorithm-based labelers) can be utilized together and mutually benefit the learning of classification tasks. A meta-training based label-sampling module is designed to attend the labels that benefit the model learning the most through additional back-propagation processes.
  arXiv  Detail & Related papers  (2020-09-22T05:38:44Z)
• UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation [52.487469544343305]
  Methods for object detection and segmentation rely on large scale instance-level annotations for training. We propose an intuitive and unified semi-supervised model that is applicable to a range of supervision.
  arXiv  Detail & Related papers  (2020-06-12T22:45:47Z)
• Global Multiclass Classification and Dataset Construction via Heterogeneous Local Experts [37.27708297562079]
  We show how to minimize the number of labelers while ensuring the reliability of the resulting dataset. Experiments with the MNIST and CIFAR-10 datasets demonstrate the favorable accuracy of our aggregation scheme.
  arXiv  Detail & Related papers  (2020-05-21T18:07:42Z)
• Interaction Matching for Long-Tail Multi-Label Classification [57.262792333593644]
  We present an elegant and effective approach for addressing limitations in existing multi-label classification models. By performing soft n-gram interaction matching, we match labels with natural language descriptions.
  arXiv  Detail & Related papers  (2020-05-18T15:27:55Z)
• Unsupervised Person Re-identification via Multi-label Classification [55.65870468861157]
  This paper formulates unsupervised person ReID as a multi-label classification task to progressively seek true labels. Our method starts by assigning each person image with a single-class label, then evolves to multi-label classification by leveraging the updated ReID model for label prediction. To boost the ReID model training efficiency in multi-label classification, we propose the memory-based multi-label classification loss (MMCL)
  arXiv  Detail & Related papers  (2020-04-20T12:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.