Modeling Text-Label Alignment for Hierarchical Text Classification

• URL: http://arxiv.org/abs/2409.00788v1
• Date: Sun, 1 Sep 2024 17:48:29 GMT
• Title: Modeling Text-Label Alignment for Hierarchical Text Classification
• Authors: Ashish Kumar, Durga Toshniwal,
• Abstract summary: Hierarchical Text Classification (HTC) aims to categorize text data based on a structured label hierarchy, resulting in predicted labels forming a sub-hierarchy tree. With the sub-hierarchy changing for each sample, the dynamic nature of text-label alignment poses challenges for existing methods. We propose a Text-Label Alignment (TLA) loss specifically designed to model the alignment between text and labels.
• Score: 12.579592946863762
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hierarchical Text Classification (HTC) aims to categorize text data based on a structured label hierarchy, resulting in predicted labels forming a sub-hierarchy tree. The semantics of the text should align with the semantics of the labels in this sub-hierarchy. With the sub-hierarchy changing for each sample, the dynamic nature of text-label alignment poses challenges for existing methods, which typically process text and labels independently. To overcome this limitation, we propose a Text-Label Alignment (TLA) loss specifically designed to model the alignment between text and labels. We obtain a set of negative labels for a given text and its positive label set. By leveraging contrastive learning, the TLA loss pulls the text closer to its positive label and pushes it away from its negative label in the embedding space. This process aligns text representations with related labels while distancing them from unrelated ones. Building upon this framework, we introduce the Hierarchical Text-Label Alignment (HTLA) model, which leverages BERT as the text encoder and GPTrans as the graph encoder and integrates text-label embeddings to generate hierarchy-aware representations. Experimental results on benchmark datasets and comparison with existing baselines demonstrate the effectiveness of HTLA for HTC.

Related papers

• HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification [19.12354692458442]
  Hierarchical text classification (HTC) is a complex subtask under multi-label text classification. We propose HiGen, a text-generation-based framework utilizing language models to encode dynamic text representations.
  arXiv  Detail & Related papers  (2024-01-24T04:44:42Z)
• MatchXML: An Efficient Text-label Matching Framework for Extreme Multi-label Text Classification [13.799733640048672]
  The eXtreme Multi-label text Classification(XMC) refers to training a classifier that assigns a text sample with relevant labels from a large-scale label set. We propose MatchXML, an efficient text-label matching framework for XMC. Experimental results demonstrate that MatchXML achieves state-of-the-art accuracy on five out of six datasets.
  arXiv  Detail & Related papers  (2023-08-25T02:32:36Z)
• Exploiting Dynamic and Fine-grained Semantic Scope for Extreme Multi-label Text Classification [12.508006325140949]
  Extreme multi-label text classification (XMTC) refers to the problem of tagging a given text with the most relevant subset of labels from a large label set. Most existing XMTC methods take advantage of fixed label clusters obtained in early stage to balance performance on tail labels and head labels. We propose a novel framework TReaderXML for XMTC, which adopts dynamic and fine-grained semantic scope from teacher knowledge.
  arXiv  Detail & Related papers  (2022-05-24T11:15:35Z)
• Label Semantic Aware Pre-training for Few-shot Text Classification [53.80908620663974]
  We propose Label Semantic Aware Pre-training (LSAP) to improve the generalization and data efficiency of text classification systems. LSAP incorporates label semantics into pre-trained generative models (T5 in our case) by performing secondary pre-training on labeled sentences from a variety of domains.
  arXiv  Detail & Related papers  (2022-04-14T17:33:34Z)
• Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification [23.719121637849806]
  We propose a hierarchy-guided Contrastive Learning (HGCLR) to embed the label hierarchy into a text encoder. During training, HGCLR constructs positive samples for input text under the guidance of the label hierarchy. After training, the HGCLR enhanced text encoder can dispense with the redundant hierarchy.
  arXiv  Detail & Related papers  (2022-03-08T03:21:45Z)
• Structured Semantic Transfer for Multi-Label Recognition with Partial Labels [85.6967666661044]
  We propose a structured semantic transfer (SST) framework that enables training multi-label recognition models with partial labels. The framework consists of two complementary transfer modules that explore within-image and cross-image semantic correlations. Experiments on the Microsoft COCO, Visual Genome and Pascal VOC datasets show that the proposed SST framework obtains superior performance over current state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-12-21T02:15:01Z)
• HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization [75.45291796263103]
  The current state-of-the-art model HiAGM for hierarchical text classification has two limitations. It correlates each text sample with all labels in the dataset which contains irrelevant information. We propose HTCInfoMax to address these issues by introducing information which includes two modules.
  arXiv  Detail & Related papers  (2021-04-12T06:04:20Z)
• MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
  MATCH is an end-to-end framework that leverages both metadata and hierarchy information. We propose different ways to regularize the parameters and output probability of each child label by its parents. Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
  arXiv  Detail & Related papers  (2021-02-15T05:23:08Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)
• Label-Wise Document Pre-Training for Multi-Label Text Classification [14.439051753832032]
  This paper develops Label-Wise Pre-Training (LW-PT) method to get a document representation with label-aware information. The basic idea is that, a multi-label document can be represented as a combination of multiple label-wise representations, and that, correlated labels always cooccur in the same or similar documents.
  arXiv  Detail & Related papers  (2020-08-15T10:34:27Z)
• Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network [61.94394163309688]
  We propose a Label-enhanced Task-Adaptive Projection Network (L-TapNet) based on the state-of-the-art few-shot classification model -- TapNet. Experimental results show that our model significantly outperforms the strongest few-shot learning baseline by 14.64 F1 scores in the one-shot setting.
  arXiv  Detail & Related papers  (2020-06-10T07:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.