Long-tailed Extreme Multi-label Text Classification with Generated
Pseudo Label Descriptions
- URL: http://arxiv.org/abs/2204.00958v1
- Date: Sat, 2 Apr 2022 23:42:32 GMT
- Title: Long-tailed Extreme Multi-label Text Classification with Generated
Pseudo Label Descriptions
- Authors: Ruohong Zhang, Yau-Shian Wang, Yiming Yang, Donghan Yu, Tom Vu, Likun
Lei
- Abstract summary: This paper addresses the challenge of tail label prediction by proposing a novel approach.
It combines the effectiveness of a trained bag-of-words (BoW) classifier in generating informative label descriptions under severe data scarce conditions.
The proposed approach achieves state-of-the-art performance on XMTC benchmark datasets and significantly outperforms the best methods so far in the tail label prediction.
- Score: 28.416742933744942
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Extreme Multi-label Text Classification (XMTC) has been a tough challenge in
machine learning research and applications due to the sheer sizes of the label
spaces and the severe data scarce problem associated with the long tail of rare
labels in highly skewed distributions. This paper addresses the challenge of
tail label prediction by proposing a novel approach, which combines the
effectiveness of a trained bag-of-words (BoW) classifier in generating
informative label descriptions under severe data scarce conditions, and the
power of neural embedding based retrieval models in mapping input documents (as
queries) to relevant label descriptions. The proposed approach achieves
state-of-the-art performance on XMTC benchmark datasets and significantly
outperforms the best methods so far in the tail label prediction. We also
provide a theoretical analysis for relating the BoW and neural models w.r.t.
performance lower bound.
Related papers
- RankMatch: A Novel Approach to Semi-Supervised Label Distribution
Learning Leveraging Inter-label Correlations [52.549807652527306]
This paper introduces RankMatch, an innovative approach for Semi-Supervised Label Distribution Learning (SSLDL)
RankMatch effectively utilizes a small number of labeled examples in conjunction with a larger quantity of unlabeled data.
We establish a theoretical generalization bound for RankMatch, and through extensive experiments, demonstrate its superiority in performance against existing SSLDL methods.
arXiv Detail & Related papers (2023-12-11T12:47:29Z) - Automated Labeling of German Chest X-Ray Radiology Reports using Deep
Learning [50.591267188664666]
We propose a deep learning-based CheXpert label prediction model, pre-trained on reports labeled by a rule-based German CheXpert model.
Our results demonstrate the effectiveness of our approach, which significantly outperformed the rule-based model on all three tasks.
arXiv Detail & Related papers (2023-06-09T16:08:35Z) - Retrieval-augmented Multi-label Text Classification [20.100081284294973]
Multi-label text classification is a challenging task in settings of large label sets.
Retrieval augmentation aims to improve the sample efficiency of classification models.
We evaluate this approach on four datasets from the legal and biomedical domains.
arXiv Detail & Related papers (2023-05-22T14:16:23Z) - Deep Partial Multi-Label Learning with Graph Disambiguation [27.908565535292723]
We propose a novel deep Partial multi-Label model with grAph-disambIguatioN (PLAIN)
Specifically, we introduce the instance-level and label-level similarities to recover label confidences.
At each training epoch, labels are propagated on the instance and label graphs to produce relatively accurate pseudo-labels.
arXiv Detail & Related papers (2023-05-10T04:02:08Z) - Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label
Learning [97.88458953075205]
Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data.
This paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner.
arXiv Detail & Related papers (2023-05-04T12:52:18Z) - Boosting Semi-Supervised Face Recognition with Noise Robustness [54.342992887966616]
This paper presents an effective solution to semi-supervised face recognition that is robust to the label noise aroused by the auto-labelling.
We develop a semi-supervised face recognition solution, named Noise Robust Learning-Labelling (NRoLL), which is based on the robust training ability empowered by GN.
arXiv Detail & Related papers (2021-05-10T14:43:11Z) - The Weakly-Labeled Rand Index [2.989889278970106]
SAS surveys produce imagery with large regions of transition between seabed types.
It is difficult to label and segment the imagery and, furthermore, challenging to score the image segmentations appropriately.
arXiv Detail & Related papers (2021-03-08T16:21:15Z) - GNN-XML: Graph Neural Networks for Extreme Multi-label Text
Classification [23.79498916023468]
Extreme multi-label text classification (XMTC) aims to tag a text instance with the most relevant subset of labels from an extremely large label set.
GNN-XML is a scalable graph neural network framework tailored for XMTC problems.
arXiv Detail & Related papers (2020-12-10T18:18:34Z) - Delving Deep into Label Smoothing [112.24527926373084]
Label smoothing is an effective regularization tool for deep neural networks (DNNs)
We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category.
arXiv Detail & Related papers (2020-11-25T08:03:11Z) - PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data.
We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
arXiv Detail & Related papers (2020-10-19T17:59:30Z) - Pretrained Generalized Autoregressive Model with Adaptive Probabilistic
Label Clusters for Extreme Multi-label Text Classification [24.665469885904145]
We propose a novel deep learning method called APLC-XLNet.
Our approach fine-tunes the recently released generalized autoregressive pretrained model (XLNet) to learn a dense representation for the input text.
Our experiments, carried out on five benchmark datasets, show that our approach has achieved new state-of-the-art results.
arXiv Detail & Related papers (2020-07-05T20:19:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.