Cluster-Guided Label Generation in Extreme Multi-Label Classification

• URL: http://arxiv.org/abs/2302.09150v1
• Date: Fri, 17 Feb 2023 21:20:36 GMT
• Title: Cluster-Guided Label Generation in Extreme Multi-Label Classification
• Authors: Taehee Jung, Joo-Kyung Kim, Sungjin Lee, and Dongyeop Kang
• Abstract summary: We cast XMC as a generation task (XLGen), where we benefit from pre-trained text-to-text models. We propose to guide label generation using label cluster information to hierarchically generate lower-level labels. XLGen with cluster guidance significantly outperforms the classification and generation baselines on tail labels.
• Score: 20.242405689985667
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For extreme multi-label classification (XMC), existing classification-based models poorly perform for tail labels and often ignore the semantic relations among labels, like treating "Wikipedia" and "Wiki" as independent and separate labels. In this paper, we cast XMC as a generation task (XLGen), where we benefit from pre-trained text-to-text models. However, generating labels from the extremely large label space is challenging without any constraints or guidance. We, therefore, propose to guide label generation using label cluster information to hierarchically generate lower-level labels. We also find that frequency-based label ordering and using decoding ensemble methods are critical factors for the improvements in XLGen. XLGen with cluster guidance significantly outperforms the classification and generation baselines on tail labels, and also generally improves the overall performance in four popular XMC benchmarks. In human evaluation, we also find XLGen generates unseen but plausible labels. Our code is now available at https://github.com/alexa/xlgen-eacl-2023.

Related papers

• Active Generalized Category Discovery [60.69060965936214]
  Generalized Category Discovery (GCD) endeavors to cluster unlabeled samples from both novel and old classes. We take the spirit of active learning and propose a new setting called Active Generalized Category Discovery (AGCD) Our method achieves state-of-the-art performance on both generic and fine-grained datasets.
  arXiv  Detail & Related papers  (2024-03-07T07:12:24Z)
• Bridging the Gap between Model Explanations in Partially Annotated Multi-label Classification [85.76130799062379]
  We study how false negative labels affect the model's explanation. We propose to boost the attribution scores of the model trained with partial labels to make its explanation resemble that of the model trained with full labels.
  arXiv  Detail & Related papers  (2023-04-04T14:00:59Z)
• Review of Extreme Multilabel Classification [1.888738346075831]
  Extreme multilabel classification or XML, is an active area of interest in machine learning. The community has come up with a useful set of metrics to identify correctly the prediction for head or tail labels.
  arXiv  Detail & Related papers  (2023-02-12T18:29:20Z)
• Pairwise Instance Relation Augmentation for Long-tailed Multi-label Text Classification [38.66674700075432]
  We propose a Pairwise Instance Relation Augmentation Network (PIRAN) to augment tailed-label documents for balancing tail labels and head labels. PIRAN consistently outperforms the SOTA methods, and dramatically improves the performance of tail labels.
  arXiv  Detail & Related papers  (2022-11-19T12:45:54Z)
• Multi-label Classification with High-rank and High-order Label Correlations [62.39748565407201]
  Previous methods capture the high-order label correlations mainly by transforming the label matrix to a latent label space with low-rank matrix factorization. We propose a simple yet effective method to depict the high-order label correlations explicitly, and at the same time maintain the high-rank of the label matrix. Comparative studies over twelve benchmark data sets validate the effectiveness of the proposed algorithm in multi-label classification.
  arXiv  Detail & Related papers  (2022-07-09T05:15:31Z)
• Open Vocabulary Extreme Classification Using Generative Models [24.17018785195843]
  The extreme multi-label classification (XMC) task aims at tagging content with a subset of labels from an extremely large label set. We propose GROOV, a fine-tuned seq2seq model for OXMC that generates the set of labels as a flat sequence and is trained using a novel loss independent of predicted label order. We show the efficacy of the approach, experimenting with popular XMC datasets for which GROOV is able to predict meaningful labels outside the given vocabulary while performing on par with state-of-the-art solutions for known labels.
  arXiv  Detail & Related papers  (2022-05-12T00:33:49Z)
• Label Disentanglement in Partition-based Extreme Multilabel Classification [111.25321342479491]
  We show that the label assignment problem in partition-based XMC can be formulated as an optimization problem. We show that our method can successfully disentangle multi-modal labels, leading to state-of-the-art (SOTA) results on four XMC benchmarks.
  arXiv  Detail & Related papers  (2021-06-24T03:24:18Z)
• MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
  MATCH is an end-to-end framework that leverages both metadata and hierarchy information. We propose different ways to regularize the parameters and output probability of each child label by its parents. Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
  arXiv  Detail & Related papers  (2021-02-15T05:23:08Z)
• LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification [27.80266694835677]
  Extreme Multi-label text Classification (XMC) is a task of finding the most relevant labels from a large label set. We propose LightXML, which adopts end-to-end training and dynamic negative labels sampling. In experiments, LightXML outperforms state-of-the-art methods in five extreme multi-label datasets.
  arXiv  Detail & Related papers  (2021-01-09T07:04:18Z)
• GNN-XML: Graph Neural Networks for Extreme Multi-label Text Classification [23.79498916023468]
  Extreme multi-label text classification (XMTC) aims to tag a text instance with the most relevant subset of labels from an extremely large label set. GNN-XML is a scalable graph neural network framework tailored for XMTC problems.
  arXiv  Detail & Related papers  (2020-12-10T18:18:34Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.