Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification

• URL: http://arxiv.org/abs/2112.06386v1
• Date: Mon, 13 Dec 2021 02:36:04 GMT
• Title: Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification
• Authors: Yinhua Piao, Sangseon Lee, Dohoon Lee, Sun Kim
• Abstract summary: We propose a novel GNN-based sparse structure learning model for inductive document classification. Our model collects a set of trainable edges connecting disjoint words between sentences and employs structure learning to sparsely select edges with dynamic contextual dependencies. Experiments on several real-world datasets demonstrate that the proposed model outperforms most state-of-the-art results.
• Score: 2.064612766965483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, graph neural networks (GNNs) have been widely used for document classification. However, most existing methods are based on static word co-occurrence graphs without sentence-level information, which poses three challenges:(1) word ambiguity, (2) word synonymity, and (3) dynamic contextual dependency. To address these challenges, we propose a novel GNN-based sparse structure learning model for inductive document classification. Specifically, a document-level graph is initially generated by a disjoint union of sentence-level word co-occurrence graphs. Our model collects a set of trainable edges connecting disjoint words between sentences and employs structure learning to sparsely select edges with dynamic contextual dependencies. Graphs with sparse structures can jointly exploit local and global contextual information in documents through GNNs. For inductive learning, the refined document graph is further fed into a general readout function for graph-level classification and optimization in an end-to-end manner. Extensive experiments on several real-world datasets demonstrate that the proposed model outperforms most state-of-the-art results, and reveal the necessity to learn sparse structures for each document.

Related papers

• Graph Neural Networks on Discriminative Graphs of Words [19.817473565906777]
  In this work, we explore a new Discriminative Graph of Words Graph Neural Network (DGoW-GNN) approach to classify text. We propose a new model for the graph-based classification of text, which combines a GNN and a sequence model. We evaluate our approach on seven benchmark datasets and find that it is outperformed by several state-of-the-art baseline models.
  arXiv  Detail & Related papers  (2024-10-27T15:14:06Z)
• GraphLSS: Integrating Lexical, Structural, and Semantic Features for Long Document Extractive Summarization [19.505955857963855]
  We present GraphLSS, a heterogeneous graph construction for long document extractive summarization. It defines two levels of information (words and sentences) and four types of edges (sentence semantic similarity, sentence occurrence order, word in sentence, and word semantic similarity) without any need for auxiliary learning models.
  arXiv  Detail & Related papers  (2024-10-25T23:48:59Z)
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks [72.01829954658889]
  This paper introduces the mathematical definition of this novel problem setting. We devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs. The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning.
  arXiv  Detail & Related papers  (2023-06-20T03:33:22Z)
• Word Grounded Graph Convolutional Network [24.6338889954789]
  Graph Convolutional Networks (GCNs) have shown strong performance in learning text representations for various tasks such as text classification. We propose to transform the document graph into a word graph, to decouple data samples and a GCN model by using a document-independent graph. The proposed Word-level Graph (WGraph) can not only implicitly learning word presentation with commonly-used word co-occurrences in corpora, but also incorporate extra global semantic dependency.
  arXiv  Detail & Related papers  (2023-05-10T19:56:55Z)
• SHGNN: Structure-Aware Heterogeneous Graph Neural Network [77.78459918119536]
  This paper proposes a novel Structure-Aware Heterogeneous Graph Neural Network (SHGNN) to address the above limitations. We first utilize a feature propagation module to capture the local structure information of intermediate nodes in the meta-path. Next, we use a tree-attention aggregator to incorporate the graph structure information into the aggregation module on the meta-path. Finally, we leverage a meta-path aggregator to fuse the information aggregated from different meta-paths.
  arXiv  Detail & Related papers  (2021-12-12T14:18:18Z)
• Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification [60.233529926965836]
  We propose a new method called SHINE, which is based on graph neural network (GNN) for short text classification. First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs. Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts.
  arXiv  Detail & Related papers  (2021-10-30T05:33:05Z)
• GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [53.70520466556453]
  We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow. In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
  arXiv  Detail & Related papers  (2021-05-06T12:20:41Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• Neural Topic Modeling by Incorporating Document Relationship Graph [18.692100955163713]
  Graph Topic Model (GTM) is a GNN based neural topic model that represents a corpus as a document relationship graph. Documents and words in the corpus become nodes in the graph and are connected based on document-word co-occurrences.
  arXiv  Detail & Related papers  (2020-09-29T12:45:55Z)
• Heterogeneous Graph Neural Networks for Extractive Document Summarization [101.17980994606836]
  Cross-sentence relations are a crucial step in extractive document summarization. We present a graph-based neural network for extractive summarization (HeterSumGraph) We introduce different types of nodes into graph-based neural networks for extractive document summarization.
  arXiv  Detail & Related papers  (2020-04-26T14:38:11Z)
• Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks [22.91359631452695]
  We propose TextING for inductive text classification via Graph Neural Networks (GNN) We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures. Our method outperforms state-of-the-art text classification methods.
  arXiv  Detail & Related papers  (2020-04-22T07:23:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.