Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification

• URL: http://arxiv.org/abs/2111.00180v1
• Date: Sat, 30 Oct 2021 05:33:05 GMT
• Title: Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification
• Authors: Yaqing Wang and Song Wang and Quanming Yao and Dejing Dou
• Abstract summary: We propose a new method called SHINE, which is based on graph neural network (GNN) for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts.
• Score: 60.233529926965836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Short text classification is a fundamental task in natural language processing. It is hard due to the lack of context information and labeled data in practice. In this paper, we propose a new method called SHINE, which is based on graph neural network (GNN), for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs which introduce more semantic and syntactic information. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts. Thus, compared with existing GNN-based methods, SHINE can better exploit interactions between nodes of the same types and capture similarities between short texts. Extensive experiments on various benchmark short text datasets show that SHINE consistently outperforms state-of-the-art methods, especially with fewer labels.

Related papers

• Pretraining Language Models with Text-Attributed Heterogeneous Graphs [28.579509154284448]
  We present a new pretraining framework for Language Models (LMs) that explicitly considers the topological and heterogeneous information in Text-Attributed Heterogeneous Graphs (TAHGs) We propose a topology-aware pretraining task to predict nodes involved in the context graph by jointly optimizing an LM and an auxiliary heterogeneous graph neural network. We conduct link prediction and node classification tasks on three datasets from various domains.
  arXiv  Detail & Related papers  (2023-10-19T08:41:21Z)
• Description-Enhanced Label Embedding Contrastive Learning for Text Classification [65.01077813330559]
  Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task. Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
  arXiv  Detail & Related papers  (2023-06-15T02:19:34Z)
• ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings [20.25180279903009]
  We propose Contrastive Graph-Text pretraining (ConGraT) for jointly learning separate representations of texts and nodes in a text-attributed graph (TAG) Our method trains a language model (LM) and a graph neural network (GNN) to align their representations in a common latent space using a batch-wise contrastive learning objective inspired by CLIP. Experiments demonstrate that ConGraT outperforms baselines on various downstream tasks, including node and text category classification, link prediction, and language modeling.
  arXiv  Detail & Related papers  (2023-05-23T17:53:30Z)
• Improving Graph-Based Text Representations with Character and Word Level N-grams [30.699644290131044]
  We propose a new word-character text graph that combines word and character n-gram nodes together with document nodes. We also propose two new graph-based neural models, WCTextGCN and WCTextGAT, for modeling our proposed text graph.
  arXiv  Detail & Related papers  (2022-10-12T08:07:54Z)
• A semantic hierarchical graph neural network for text classification [1.439766998338892]
  We propose a new hierarchical graph neural network (HieGNN) which extracts corresponding information from word-level, sentence-level and document-level respectively. Experimental results on several benchmark datasets achieve better or similar results compared to several baseline methods.
  arXiv  Detail & Related papers  (2022-09-15T03:59:31Z)
• Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification [2.064612766965483]
  We propose a novel GNN-based sparse structure learning model for inductive document classification. Our model collects a set of trainable edges connecting disjoint words between sentences and employs structure learning to sparsely select edges with dynamic contextual dependencies. Experiments on several real-world datasets demonstrate that the proposed model outperforms most state-of-the-art results.
  arXiv  Detail & Related papers  (2021-12-13T02:36:04Z)
• GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [53.70520466556453]
  We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow. In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
  arXiv  Detail & Related papers  (2021-05-06T12:20:41Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• Be More with Less: Hypergraph Attention Networks for Inductive Text Classification [56.98218530073927]
  Graph neural networks (GNNs) have received increasing attention in the research community and demonstrated their promising results on this canonical task. Despite the success, their performance could be largely jeopardized in practice since they are unable to capture high-order interaction between words. We propose a principled model -- hypergraph attention networks (HyperGAT) which can obtain more expressive power with less computational consumption for text representation learning.
  arXiv  Detail & Related papers  (2020-11-01T00:21:59Z)
• Exploring the Hierarchy in Relation Labels for Scene Graph Generation [75.88758055269948]
  The proposed method can improve several state-of-the-art baselines by a large margin (up to $33%$ relative gain) in terms of Recall@50. Experiments show that the proposed simple yet effective method can improve several state-of-the-art baselines by a large margin.
  arXiv  Detail & Related papers  (2020-09-12T17:36:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.