LA-HCN: Label-based Attention for Hierarchical Multi-label TextClassification Neural Network

• URL: http://arxiv.org/abs/2009.10938v3
• Date: Sat, 10 Apr 2021 12:20:53 GMT
• Title: LA-HCN: Label-based Attention for Hierarchical Multi-label TextClassification Neural Network
• Authors: Xinyi Zhang and Jiahao Xu and Charlie Soh and Lihui Chen
• Abstract summary: We propose a Label-based Attention for Hierarchical Mutlti-label Text Classification Neural Network (LA-HCN) LA-HCN is designed to hierarchically extract important information from the text based on the labels from different hierarchy levels. Separate local and global document embeddings are obtained and used to facilitate the respective local and global classifications.
• Score: 16.12197413284402
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hierarchical multi-label text classification (HMTC) has been gaining popularity in recent years thanks to its applicability to a plethora of real-world applications. The existing HMTC algorithms largely focus on the design of classifiers, such as the local, global, or a combination of them. However, very few studies have focused on hierarchical feature extraction and explore the association between the hierarchical labels and the text. In this paper, we propose a Label-based Attention for Hierarchical Mutlti-label Text Classification Neural Network (LA-HCN), where the novel label-based attention module is designed to hierarchically extract important information from the text based on the labels from different hierarchy levels. Besides, hierarchical information is shared across levels while preserving the hierarchical label-based information. Separate local and global document embeddings are obtained and used to facilitate the respective local and global classifications. In our experiments, LA-HCN outperforms other state-of-the-art neural network-based HMTC algorithms on four public HMTC datasets. The ablation study also demonstrates the effectiveness of the proposed label-based attention module as well as the novel local and global embeddings and classifications. By visualizing the learned attention (words), we find that LA-HCN is able to extract meaningful information corresponding to the different labels which provides explainability that may be helpful for the human analyst.

Related papers

• Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification [9.391704905671476]
  This paper aims to rethink the data challenges in medical texts and present a novel framework-agnostic algorithm called Text2Tree. We embed the ICD code tree structure of labels into cascade attention modules for learning hierarchy-aware label representations. Two new learning schemes, Similarity Surrogate Learning (SSL) and Dissimilarity Mixup Learning (DML), are devised to boost text classification by reusing and distinguishing samples of other labels.
  arXiv  Detail & Related papers  (2023-11-28T10:02:08Z)
• Description-Enhanced Label Embedding Contrastive Learning for Text Classification [65.01077813330559]
  Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task. Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
  arXiv  Detail & Related papers  (2023-06-15T02:19:34Z)
• SEAL: Simultaneous Label Hierarchy Exploration And Learning [9.701914280306118]
  We propose a new framework that explores the label hierarchy by augmenting the observed labels with latent labels that follow a prior hierarchical structure. Our approach uses a 1-Wasserstein metric over the tree metric space as an objective function, which enables us to simultaneously learn a data-driven label hierarchy and perform (semi-supervised) learning.
  arXiv  Detail & Related papers  (2023-04-26T08:31:59Z)
• Exploiting Global and Local Hierarchies for Hierarchical Text Classification [34.624922210257125]
  Existing methods encode label hierarchy in a global view, where label hierarchy is treated as the static hierarchical structure containing all labels. We propose Hierarchy-guided BERT with Global and Local hierarchies (HBGL) to model both global and local hierarchies. Compared with the state-of-the-art method HGCLR, our method achieves significant improvement on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-05-05T12:48:41Z)
• Deep Hierarchical Semantic Segmentation [76.40565872257709]
  hierarchical semantic segmentation (HSS) aims at structured, pixel-wise description of visual observation in terms of a class hierarchy. HSSN casts HSS as a pixel-wise multi-label classification task, only bringing minimal architecture change to current segmentation models. With hierarchy-induced margin constraints, HSSN reshapes the pixel embedding space, so as to generate well-structured pixel representations.
  arXiv  Detail & Related papers  (2022-03-27T15:47:44Z)
• MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
  MATCH is an end-to-end framework that leverages both metadata and hierarchy information. We propose different ways to regularize the parameters and output probability of each child label by its parents. Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
  arXiv  Detail & Related papers  (2021-02-15T05:23:08Z)
• An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels [49.036212158261215]
  Large-scale Multi-label Text Classification (LMTC) has a wide range of Natural Language Processing (NLP) applications. Current state-of-the-art LMTC models employ Label-Wise Attention Networks (LWANs) We show that hierarchical methods based on Probabilistic Label Trees (PLTs) outperform LWANs. We propose a new state-of-the-art method which combines BERT with LWANs.
  arXiv  Detail & Related papers  (2020-10-04T18:55:47Z)
• Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition [75.44233392355711]
  KGGR framework exploits prior knowledge of statistical label correlations with deep neural networks. It first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features. It exploits a graph propagation network to explore graph node interactions.
  arXiv  Detail & Related papers  (2020-09-20T15:05:29Z)
• Multi-Level Graph Convolutional Network with Automatic Graph Learning for Hyperspectral Image Classification [63.56018768401328]
  We propose a Multi-level Graph Convolutional Network (GCN) with Automatic Graph Learning method (MGCN-AGL) for HSI classification. By employing attention mechanism to characterize the importance among spatially neighboring regions, the most relevant information can be adaptively incorporated to make decisions. Our MGCN-AGL encodes the long range dependencies among image regions based on the expressive representations that have been produced at local level.
  arXiv  Detail & Related papers  (2020-09-19T09:26:20Z)
• Joint Embedding of Words and Category Labels for Hierarchical Multi-label Text Classification [4.2750700546937335]
  hierarchical text classification (HTC) has received extensive attention and has broad application prospects. We propose a joint embedding of text and parent category based on hierarchical fine-tuning ordered neurons LSTM (HFT-ONLSTM) for HTC.
  arXiv  Detail & Related papers  (2020-04-06T11:06:08Z)
• An Ontology-Aware Framework for Audio Event Classification [19.11706899266862]
  Recent advancements in audio event classification often ignore the structure and relation between the label classes available as prior information. We propose an ontology-aware neural network containing two components: feed-forward ontology layers and graph convolutional networks (GCN) The framework is evaluated on two benchmark datasets for single-label and multi-label audio event classification tasks.
  arXiv  Detail & Related papers  (2020-01-27T20:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.