Label Mask for Multi-Label Text Classification

• URL: http://arxiv.org/abs/2106.10076v1
• Date: Fri, 18 Jun 2021 11:54:33 GMT
• Title: Label Mask for Multi-Label Text Classification
• Authors: Rui Song, Xingbing Chen, Zelong Liu, Haining An, Zhiqi Zhang, Xiaoguang Wang, Hao Xu
• Abstract summary: We propose a Label Mask multi-label text classification model (LM-MTC), which is inspired by the idea of cloze questions of language model. On the basis, we assign a different token to each potential label, and randomly mask the token with a certain probability to build a label based Masked Language Model (MLM)
• Score: 6.742627397194543
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the key problems in multi-label text classification is how to take advantage of the correlation among labels. However, it is very challenging to directly model the correlations among labels in a complex and unknown label space. In this paper, we propose a Label Mask multi-label text classification model (LM-MTC), which is inspired by the idea of cloze questions of language model. LM-MTC is able to capture implicit relationships among labels through the powerful ability of pre-train language models. On the basis, we assign a different token to each potential label, and randomly mask the token with a certain probability to build a label based Masked Language Model (MLM). We train the MTC and MLM together, further improving the generalization ability of the model. A large number of experiments on multiple datasets demonstrate the effectiveness of our method.

Related papers

• Deep Partial Multi-Label Learning with Graph Disambiguation [27.908565535292723]
  We propose a novel deep Partial multi-Label model with grAph-disambIguatioN (PLAIN) Specifically, we introduce the instance-level and label-level similarities to recover label confidences. At each training epoch, labels are propagated on the instance and label graphs to produce relatively accurate pseudo-labels.
  arXiv  Detail & Related papers  (2023-05-10T04:02:08Z)
• Complementary to Multiple Labels: A Correlation-Aware Correction Approach [65.59584909436259]
  We show theoretically how the estimated transition matrix in multi-class CLL could be distorted in multi-labeled cases. We propose a two-step method to estimate the transition matrix from candidate labels.
  arXiv  Detail & Related papers  (2023-02-25T04:48:48Z)
• Label2Label: A Language Modeling Framework for Multi-Attribute Learning [93.68058298766739]
  Label2Label is the first attempt for multi-attribute prediction from the perspective of language modeling. Inspired by the success of pre-training language models in NLP, Label2Label introduces an image-conditioned masked language model. Our intuition is that the instance-wise attribute relations are well grasped if the neural net can infer the missing attributes based on the context and the remaining attribute hints.
  arXiv  Detail & Related papers  (2022-07-18T15:12:33Z)
• Large Loss Matters in Weakly Supervised Multi-Label Classification [50.262533546999045]
  We first regard unobserved labels as negative labels, casting the W task into noisy multi-label classification. We propose novel methods for W which reject or correct the large loss samples to prevent model from memorizing the noisy label. Our methodology actually works well, validating that treating large loss properly matters in a weakly supervised multi-label classification.
  arXiv  Detail & Related papers  (2022-06-08T08:30:24Z)
• One Positive Label is Sufficient: Single-Positive Multi-Label Learning with Label Enhancement [71.9401831465908]
  We investigate single-positive multi-label learning (SPMLL) where each example is annotated with only one relevant label. A novel method named proposed, i.e., Single-positive MultI-label learning with Label Enhancement, is proposed. Experiments on benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-06-01T14:26:30Z)
• GuidedMix-Net: Learning to Improve Pseudo Masks Using Labeled Images as Reference [153.354332374204]
  We propose a novel method for semi-supervised semantic segmentation named GuidedMix-Net. We first introduce a feature alignment objective between labeled and unlabeled data to capture potentially similar image pairs. MITrans is shown to be a powerful knowledge module for further progressive refining features of unlabeled data. Along with supervised learning for labeled data, the prediction of unlabeled data is jointly learned with the generated pseudo masks.
  arXiv  Detail & Related papers  (2021-06-29T02:48:45Z)
• Label Confusion Learning to Enhance Text Classification Models [3.0251266104313643]
  Label Confusion Model (LCM) learns label confusion to capture semantic overlap among labels. LCM can generate a better label distribution to replace the original one-hot label vector. experiments on five text classification benchmark datasets reveal the effectiveness of LCM for several widely used deep learning classification models.
  arXiv  Detail & Related papers  (2020-12-09T11:34:35Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)
• Label-Wise Document Pre-Training for Multi-Label Text Classification [14.439051753832032]
  This paper develops Label-Wise Pre-Training (LW-PT) method to get a document representation with label-aware information. The basic idea is that, a multi-label document can be represented as a combination of multiple label-wise representations, and that, correlated labels always cooccur in the same or similar documents.
  arXiv  Detail & Related papers  (2020-08-15T10:34:27Z)
• Multi-Label Text Classification using Attention-based Graph Neural Network [0.0]
  A graph attention network-based model is proposed to capture the attentive dependency structure among the labels. The proposed model achieves similar or better performance compared to the previous state-of-the-art models.
  arXiv  Detail & Related papers  (2020-03-22T17:12:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.