Long-tailed Extreme Multi-label Text Classification with Generated Pseudo Label Descriptions

• URL: http://arxiv.org/abs/2204.00958v1
• Date: Sat, 2 Apr 2022 23:42:32 GMT
• Title: Long-tailed Extreme Multi-label Text Classification with Generated Pseudo Label Descriptions
• Authors: Ruohong Zhang, Yau-Shian Wang, Yiming Yang, Donghan Yu, Tom Vu, Likun Lei
• Abstract summary: This paper addresses the challenge of tail label prediction by proposing a novel approach. It combines the effectiveness of a trained bag-of-words (BoW) classifier in generating informative label descriptions under severe data scarce conditions. The proposed approach achieves state-of-the-art performance on XMTC benchmark datasets and significantly outperforms the best methods so far in the tail label prediction.
• Score: 28.416742933744942
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extreme Multi-label Text Classification (XMTC) has been a tough challenge in machine learning research and applications due to the sheer sizes of the label spaces and the severe data scarce problem associated with the long tail of rare labels in highly skewed distributions. This paper addresses the challenge of tail label prediction by proposing a novel approach, which combines the effectiveness of a trained bag-of-words (BoW) classifier in generating informative label descriptions under severe data scarce conditions, and the power of neural embedding based retrieval models in mapping input documents (as queries) to relevant label descriptions. The proposed approach achieves state-of-the-art performance on XMTC benchmark datasets and significantly outperforms the best methods so far in the tail label prediction. We also provide a theoretical analysis for relating the BoW and neural models w.r.t. performance lower bound.

Related papers

• Prototypical Extreme Multi-label Classification with a Dynamic Margin Loss [6.244642999033755]
  Extreme Multi-label Classification (XMC) methods predict relevant labels for a given query in an extremely large label space. Recent works in XMC address this problem using deep encoders that project text descriptions to an embedding space suitable for recovering the closest labels. We propose PRIME, a XMC method that employs a novel prototypical contrastive learning technique to reconcile efficiency and performance surpassing brute-force approaches.
  arXiv  Detail & Related papers  (2024-10-27T10:24:23Z)
• Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
  Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
  arXiv  Detail & Related papers  (2024-10-08T15:06:10Z)
• Automated Labeling of German Chest X-Ray Radiology Reports using Deep Learning [50.591267188664666]
  We propose a deep learning-based CheXpert label prediction model, pre-trained on reports labeled by a rule-based German CheXpert model. Our results demonstrate the effectiveness of our approach, which significantly outperformed the rule-based model on all three tasks.
  arXiv  Detail & Related papers  (2023-06-09T16:08:35Z)
• Retrieval-augmented Multi-label Text Classification [20.100081284294973]
  Multi-label text classification is a challenging task in settings of large label sets. Retrieval augmentation aims to improve the sample efficiency of classification models. We evaluate this approach on four datasets from the legal and biomedical domains.
  arXiv  Detail & Related papers  (2023-05-22T14:16:23Z)
• Deep Partial Multi-Label Learning with Graph Disambiguation [27.908565535292723]
  We propose a novel deep Partial multi-Label model with grAph-disambIguatioN (PLAIN) Specifically, we introduce the instance-level and label-level similarities to recover label confidences. At each training epoch, labels are propagated on the instance and label graphs to produce relatively accurate pseudo-labels.
  arXiv  Detail & Related papers  (2023-05-10T04:02:08Z)
• Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning [97.88458953075205]
  Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. This paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner.
  arXiv  Detail & Related papers  (2023-05-04T12:52:18Z)
• Boosting Semi-Supervised Face Recognition with Noise Robustness [54.342992887966616]
  This paper presents an effective solution to semi-supervised face recognition that is robust to the label noise aroused by the auto-labelling. We develop a semi-supervised face recognition solution, named Noise Robust Learning-Labelling (NRoLL), which is based on the robust training ability empowered by GN.
  arXiv  Detail & Related papers  (2021-05-10T14:43:11Z)
• GNN-XML: Graph Neural Networks for Extreme Multi-label Text Classification [23.79498916023468]
  Extreme multi-label text classification (XMTC) aims to tag a text instance with the most relevant subset of labels from an extremely large label set. GNN-XML is a scalable graph neural network framework tailored for XMTC problems.
  arXiv  Detail & Related papers  (2020-12-10T18:18:34Z)
• Delving Deep into Label Smoothing [112.24527926373084]
  Label smoothing is an effective regularization tool for deep neural networks (DNNs) We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category.
  arXiv  Detail & Related papers  (2020-11-25T08:03:11Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)
• Pretrained Generalized Autoregressive Model with Adaptive Probabilistic Label Clusters for Extreme Multi-label Text Classification [24.665469885904145]
  We propose a novel deep learning method called APLC-XLNet. Our approach fine-tunes the recently released generalized autoregressive pretrained model (XLNet) to learn a dense representation for the input text. Our experiments, carried out on five benchmark datasets, show that our approach has achieved new state-of-the-art results.
  arXiv  Detail & Related papers  (2020-07-05T20:19:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.