Light-weight Deep Extreme Multilabel Classification

• URL: http://arxiv.org/abs/2304.11045v1
• Date: Thu, 20 Apr 2023 09:06:10 GMT
• Title: Light-weight Deep Extreme Multilabel Classification
• Authors: Istasis Mishra, Arpan Dasgupta, Pratik Jawanpuria, Bamdev Mishra, and Pawan Kumar
• Abstract summary: Extreme multi-label (XML) classification refers to the task of supervised multi-label learning that involves a large number of labels. We develop a method called LightDXML which modifies the recently developed deep learning based XML framework by using label embeddings. LightDXML also removes the requirement of a re-ranker module, thereby, leading to further savings on time and memory requirements.
• Score: 12.29534534973133
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extreme multi-label (XML) classification refers to the task of supervised multi-label learning that involves a large number of labels. Hence, scalability of the classifier with increasing label dimension is an important consideration. In this paper, we develop a method called LightDXML which modifies the recently developed deep learning based XML framework by using label embeddings instead of feature embedding for negative sampling and iterating cyclically through three major phases: (1) proxy training of label embeddings (2) shortlisting of labels for negative sampling and (3) final classifier training using the negative samples. Consequently, LightDXML also removes the requirement of a re-ranker module, thereby, leading to further savings on time and memory requirements. The proposed method achieves the best of both worlds: while the training time, model size and prediction times are on par or better compared to the tree-based methods, it attains much better prediction accuracy that is on par with the deep learning based methods. Moreover, the proposed approach achieves the best tail-label prediction accuracy over most state-of-the-art XML methods on some of the large datasets\footnote{accepted in IJCNN 2023, partial funding from MAPG grant and IIIT Seed grant at IIIT, Hyderabad, India. Code: \url{https://github.com/misterpawan/LightDXML}

Related papers

• Learning from Label Proportions: Bootstrapping Supervised Learners via Belief Propagation [18.57840057487926]
  Learning from Label Proportions (LLP) is a learning problem where only aggregate level labels are available for groups of instances, called bags, during training. This setting arises in domains like advertising and medicine due to privacy considerations. We propose a novel algorithmic framework for this problem that iteratively performs two main steps.
  arXiv  Detail & Related papers  (2023-10-12T06:09:26Z)
• Substituting Data Annotation with Balanced Updates and Collective Loss in Multi-label Text Classification [19.592985329023733]
  Multi-label text classification (MLTC) is the task of assigning multiple labels to a given text. We study the MLTC problem in annotation-free and scarce-annotation settings in which the magnitude of available supervision signals is linear to the number of labels. Our method follows three steps, (1) mapping input text into a set of preliminary label likelihoods by natural language inference using a pre-trained language model, (2) calculating a signed label dependency graph by label descriptions, and (3) updating the preliminary label likelihoods with message passing along the label dependency graph.
  arXiv  Detail & Related papers  (2023-09-24T04:12:52Z)
• Multi-Label Knowledge Distillation [86.03990467785312]
  We propose a novel multi-label knowledge distillation method. On one hand, it exploits the informative semantic knowledge from the logits by dividing the multi-label learning problem into a set of binary classification problems. On the other hand, it enhances the distinctiveness of the learned feature representations by leveraging the structural information of label-wise embeddings.
  arXiv  Detail & Related papers  (2023-08-12T03:19:08Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• A Survey on Extreme Multi-label Learning [72.8751573611815]
  Multi-label learning has attracted significant attention from both academic and industry field in recent decades. It is infeasible to directly adapt them to extremely large label space because of the compute and memory overhead. eXtreme Multi-label Learning (XML) is becoming an important task and many effective approaches are proposed.
  arXiv  Detail & Related papers  (2022-10-08T08:31:34Z)
• Large Loss Matters in Weakly Supervised Multi-Label Classification [50.262533546999045]
  We first regard unobserved labels as negative labels, casting the W task into noisy multi-label classification. We propose novel methods for W which reject or correct the large loss samples to prevent model from memorizing the noisy label. Our methodology actually works well, validating that treating large loss properly matters in a weakly supervised multi-label classification.
  arXiv  Detail & Related papers  (2022-06-08T08:30:24Z)
• Large-Scale Pre-training for Person Re-identification with Noisy Labels [125.49696935852634]
  We develop a large-scale Pre-training framework utilizing Noisy Labels (PNL) In principle, joint learning of these three modules not only clusters similar examples to one prototype, but also rectifies noisy labels based on the prototype assignment. This simple pre-training task provides a scalable way to learn SOTA Re-ID representations from scratch on "LUPerson-NL" without bells and whistles.
  arXiv  Detail & Related papers  (2022-03-30T17:59:58Z)
• DECAF: Deep Extreme Classification with Label Features [9.768907751312396]
  Extreme multi-label classification (XML) involves tagging a data point with its most relevant subset of labels from an extremely large label set. Leading XML algorithms scale to millions of labels, but they largely ignore label meta-data such as textual descriptions of the labels. This paper develops the DECAF algorithm that addresses these challenges by learning models enriched by label metadata.
  arXiv  Detail & Related papers  (2021-08-01T05:36:05Z)
• LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification [27.80266694835677]
  Extreme Multi-label text Classification (XMC) is a task of finding the most relevant labels from a large label set. We propose LightXML, which adopts end-to-end training and dynamic negative labels sampling. In experiments, LightXML outperforms state-of-the-art methods in five extreme multi-label datasets.
  arXiv  Detail & Related papers  (2021-01-09T07:04:18Z)
• An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels [49.036212158261215]
  Large-scale Multi-label Text Classification (LMTC) has a wide range of Natural Language Processing (NLP) applications. Current state-of-the-art LMTC models employ Label-Wise Attention Networks (LWANs) We show that hierarchical methods based on Probabilistic Label Trees (PLTs) outperform LWANs. We propose a new state-of-the-art method which combines BERT with LWANs.
  arXiv  Detail & Related papers  (2020-10-04T18:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.