Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks

• URL: http://arxiv.org/abs/2004.13826v2
• Date: Tue, 12 May 2020 08:28:27 GMT
• Title: Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks
• Authors: Yufeng Zhang, Xueli Yu, Zeyu Cui, Shu Wu, Zhongzhen Wen and Liang Wang
• Abstract summary: We propose TextING for inductive text classification via Graph Neural Networks (GNN) We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures. Our method outperforms state-of-the-art text classification methods.
• Score: 22.91359631452695
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text classification is fundamental in natural language processing (NLP), and Graph Neural Networks (GNN) are recently applied in this task. However, the existing graph-based works can neither capture the contextual word relationships within each document nor fulfil the inductive learning of new words. In this work, to overcome such problems, we propose TextING for inductive text classification via GNN. We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures, which can also effectively produce embeddings for unseen words in the new document. Finally, the word nodes are aggregated as the document embedding. Extensive experiments on four benchmark datasets show that our method outperforms state-of-the-art text classification methods.

Related papers

• Graph Neural Networks on Discriminative Graphs of Words [19.817473565906777]
  In this work, we explore a new Discriminative Graph of Words Graph Neural Network (DGoW-GNN) approach to classify text. We propose a new model for the graph-based classification of text, which combines a GNN and a sequence model. We evaluate our approach on seven benchmark datasets and find that it is outperformed by several state-of-the-art baseline models.
  arXiv  Detail & Related papers  (2024-10-27T15:14:06Z)
• Pretraining Language Models with Text-Attributed Heterogeneous Graphs [28.579509154284448]
  We present a new pretraining framework for Language Models (LMs) that explicitly considers the topological and heterogeneous information in Text-Attributed Heterogeneous Graphs (TAHGs) We propose a topology-aware pretraining task to predict nodes involved in the context graph by jointly optimizing an LM and an auxiliary heterogeneous graph neural network. We conduct link prediction and node classification tasks on three datasets from various domains.
  arXiv  Detail & Related papers  (2023-10-19T08:41:21Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)
• Word Grounded Graph Convolutional Network [24.6338889954789]
  Graph Convolutional Networks (GCNs) have shown strong performance in learning text representations for various tasks such as text classification. We propose to transform the document graph into a word graph, to decouple data samples and a GCN model by using a document-independent graph. The proposed Word-level Graph (WGraph) can not only implicitly learning word presentation with commonly-used word co-occurrences in corpora, but also incorporate extra global semantic dependency.
  arXiv  Detail & Related papers  (2023-05-10T19:56:55Z)
• Understanding Graph Convolutional Networks for Text Classification [9.495731689143827]
  We conduct a comprehensive analysis of the role of node and edge embeddings in a graph and its GCN learning techniques in text classification. Our analysis is the first of its kind and provides useful insights into the importance of each graph node/edge construction mechanism.
  arXiv  Detail & Related papers  (2022-03-30T05:14:31Z)
• Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification [2.064612766965483]
  We propose a novel GNN-based sparse structure learning model for inductive document classification. Our model collects a set of trainable edges connecting disjoint words between sentences and employs structure learning to sparsely select edges with dynamic contextual dependencies. Experiments on several real-world datasets demonstrate that the proposed model outperforms most state-of-the-art results.
  arXiv  Detail & Related papers  (2021-12-13T02:36:04Z)
• Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification [60.233529926965836]
  We propose a new method called SHINE, which is based on graph neural network (GNN) for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts.
  arXiv  Detail & Related papers  (2021-10-30T05:33:05Z)
• Graph Neural Networks for Natural Language Processing: A Survey [64.36633422999905]
  We present a comprehensive overview onGraph Neural Networks (GNNs) for Natural Language Processing. We propose a new taxonomy of GNNs for NLP, which organizes existing research of GNNs for NLP along three axes: graph construction,graph representation learning, and graph based encoder-decoder models.
  arXiv  Detail & Related papers  (2021-06-10T23:59:26Z)
• GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [53.70520466556453]
  We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow. In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
  arXiv  Detail & Related papers  (2021-05-06T12:20:41Z)
• Be More with Less: Hypergraph Attention Networks for Inductive Text Classification [56.98218530073927]
  Graph neural networks (GNNs) have received increasing attention in the research community and demonstrated their promising results on this canonical task. Despite the success, their performance could be largely jeopardized in practice since they are unable to capture high-order interaction between words. We propose a principled model -- hypergraph attention networks (HyperGAT) which can obtain more expressive power with less computational consumption for text representation learning.
  arXiv  Detail & Related papers  (2020-11-01T00:21:59Z)
• Heterogeneous Graph Neural Networks for Extractive Document Summarization [101.17980994606836]
  Cross-sentence relations are a crucial step in extractive document summarization. We present a graph-based neural network for extractive summarization (HeterSumGraph) We introduce different types of nodes into graph-based neural networks for extractive document summarization.
  arXiv  Detail & Related papers  (2020-04-26T14:38:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.