UniDEC : Unified Dual Encoder and Classifier Training for Extreme Multi-Label Classification

• URL: http://arxiv.org/abs/2405.03714v1
• Date: Sat, 4 May 2024 17:27:51 GMT
• Title: UniDEC : Unified Dual Encoder and Classifier Training for Extreme Multi-Label Classification
• Authors: Siddhant Kharbanda, Devaansh Gupta, Gururaj K, Pankaj Malhotra, Cho-Jui Hsieh, Rohit Babbar,
• Abstract summary: Extreme Multi-label Classification (XMC) involves predicting a subset of relevant labels from an extremely large label space. This work proposes UniDEC, a novel end-to-end trainable framework which trains the dual encoder and classifier in together.
• Score: 42.36546066941635
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extreme Multi-label Classification (XMC) involves predicting a subset of relevant labels from an extremely large label space, given an input query and labels with textual features. Models developed for this problem have conventionally used modular approach with (i) a Dual Encoder (DE) to embed the queries and label texts, (ii) a One-vs-All classifier to rerank the shortlisted labels mined through meta-classifier training. While such methods have shown empirical success, we observe two key uncharted aspects, (i) DE training typically uses only a single positive relation even for datasets which offer more, (ii) existing approaches fixate on using only OvA reduction of the multi-label problem. This work aims to explore these aspects by proposing UniDEC, a novel end-to-end trainable framework which trains the dual encoder and classifier in together in a unified fashion using a multi-class loss. For the choice of multi-class loss, the work proposes a novel pick-some-label (PSL) reduction of the multi-label problem with leverages multiple (in come cases, all) positives. The proposed framework achieves state-of-the-art results on a single GPU, while achieving on par results with respect to multi-GPU SOTA methods on various XML benchmark datasets, all while using 4-16x lesser compute and being practically scalable even beyond million label scale datasets.

Related papers

• Prototypical Extreme Multi-label Classification with a Dynamic Margin Loss [6.244642999033755]
  Extreme Multi-label Classification (XMC) methods predict relevant labels for a given query in an extremely large label space. Recent works in XMC address this problem using deep encoders that project text descriptions to an embedding space suitable for recovering the closest labels. We propose PRIME, a XMC method that employs a novel prototypical contrastive learning technique to reconcile efficiency and performance surpassing brute-force approaches.
  arXiv  Detail & Related papers  (2024-10-27T10:24:23Z)
• Learning label-label correlations in Extreme Multi-label Classification via Label Features [44.00852282861121]
  Extreme Multi-label Text Classification (XMC) involves learning a classifier that can assign an input with a subset of most relevant labels from millions of label choices. Short-text XMC with label features has found numerous applications in areas such as query-to-ad-phrase matching in search ads, title-based product recommendation, prediction of related searches. We propose Gandalf, a novel approach which makes use of a label co-occurrence graph to leverage label features as additional data points to supplement the training distribution.
  arXiv  Detail & Related papers  (2024-05-03T21:18:43Z)
• Reliable Representations Learning for Incomplete Multi-View Partial Multi-Label Classification [78.15629210659516]
  In this paper, we propose an incomplete multi-view partial multi-label classification network named RANK. We break through the view-level weights inherent in existing methods and propose a quality-aware sub-network to dynamically assign quality scores to each view of each sample. Our model is not only able to handle complete multi-view multi-label datasets, but also works on datasets with missing instances and labels.
  arXiv  Detail & Related papers  (2023-03-30T03:09:25Z)
• An Effective Approach for Multi-label Classification with Missing Labels [8.470008570115146]
  We propose a pseudo-label based approach to reduce the cost of annotation without bringing additional complexity to the classification networks. By designing a novel loss function, we are able to relax the requirement that each instance must contain at least one positive label. We show that our method can handle the imbalance between positive labels and negative labels, while still outperforming existing missing-label learning approaches.
  arXiv  Detail & Related papers  (2022-10-24T23:13:57Z)
• Open Vocabulary Multi-Label Classification with Dual-Modal Decoder on Aligned Visual-Textual Features [14.334304670606633]
  We propose a novel algorithm, Aligned Dual moDality ClaSsifier (ADDS), which includes a Dual-Modal decoder (DM-decoder) with alignment between visual and textual features. Extensive experiments conducted on several standard benchmarks, NUS-WIDE, ImageNet-1k, ImageNet-21k, and MS-COCO, demonstrate that our approach significantly outperforms previous methods.
  arXiv  Detail & Related papers  (2022-08-19T22:45:07Z)
• One Positive Label is Sufficient: Single-Positive Multi-Label Learning with Label Enhancement [71.9401831465908]
  We investigate single-positive multi-label learning (SPMLL) where each example is annotated with only one relevant label. A novel method named proposed, i.e., Single-positive MultI-label learning with Label Enhancement, is proposed. Experiments on benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-06-01T14:26:30Z)
• Learning Semantic Segmentation from Multiple Datasets with Label Shifts [101.24334184653355]
  This paper proposes UniSeg, an effective approach to automatically train models across multiple datasets with differing label spaces. Specifically, we propose two losses that account for conflicting and co-occurring labels to achieve better generalization performance in unseen domains.
  arXiv  Detail & Related papers  (2022-02-28T18:55:19Z)
• Label Disentanglement in Partition-based Extreme Multilabel Classification [111.25321342479491]
  We show that the label assignment problem in partition-based XMC can be formulated as an optimization problem. We show that our method can successfully disentangle multi-modal labels, leading to state-of-the-art (SOTA) results on four XMC benchmarks.
  arXiv  Detail & Related papers  (2021-06-24T03:24:18Z)
• An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels [49.036212158261215]
  Large-scale Multi-label Text Classification (LMTC) has a wide range of Natural Language Processing (NLP) applications. Current state-of-the-art LMTC models employ Label-Wise Attention Networks (LWANs) We show that hierarchical methods based on Probabilistic Label Trees (PLTs) outperform LWANs. We propose a new state-of-the-art method which combines BERT with LWANs.
  arXiv  Detail & Related papers  (2020-10-04T18:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.