Sparse Structure Learning via Graph Neural Networks for Inductive
Document Classification
- URL: http://arxiv.org/abs/2112.06386v1
- Date: Mon, 13 Dec 2021 02:36:04 GMT
- Title: Sparse Structure Learning via Graph Neural Networks for Inductive
Document Classification
- Authors: Yinhua Piao, Sangseon Lee, Dohoon Lee, Sun Kim
- Abstract summary: We propose a novel GNN-based sparse structure learning model for inductive document classification.
Our model collects a set of trainable edges connecting disjoint words between sentences and employs structure learning to sparsely select edges with dynamic contextual dependencies.
Experiments on several real-world datasets demonstrate that the proposed model outperforms most state-of-the-art results.
- Score: 2.064612766965483
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recently, graph neural networks (GNNs) have been widely used for document
classification. However, most existing methods are based on static word
co-occurrence graphs without sentence-level information, which poses three
challenges:(1) word ambiguity, (2) word synonymity, and (3) dynamic contextual
dependency. To address these challenges, we propose a novel GNN-based sparse
structure learning model for inductive document classification. Specifically, a
document-level graph is initially generated by a disjoint union of
sentence-level word co-occurrence graphs. Our model collects a set of trainable
edges connecting disjoint words between sentences and employs structure
learning to sparsely select edges with dynamic contextual dependencies. Graphs
with sparse structures can jointly exploit local and global contextual
information in documents through GNNs. For inductive learning, the refined
document graph is further fed into a general readout function for graph-level
classification and optimization in an end-to-end manner. Extensive experiments
on several real-world datasets demonstrate that the proposed model outperforms
most state-of-the-art results, and reveal the necessity to learn sparse
structures for each document.
Related papers
- Hierarchical Attention Graph for Scientific Document Summarization in Global and Local Level [3.7651378994837104]
Long input hinders simultaneous modeling of both global high-order relations between sentences and local intra-sentence relations.
We propose HAESum, a novel approach utilizing graph neural networks to model documents based on their hierarchical discourse structure.
We validate our approach on two benchmark datasets, and the experimental results demonstrate the effectiveness of HAESum.
arXiv Detail & Related papers (2024-05-16T15:46:30Z) - Modeling Unified Semantic Discourse Structure for High-quality Headline Generation [45.23071138765902]
We propose using a unified semantic discourse structure (S3) to represent document semantics.
The hierarchical composition of sentence, clause, and word intrinsically characterizes the semantic meaning of the overall document.
Our work can be instructive for a broad range of document modeling tasks, more than headline or summarization generation.
arXiv Detail & Related papers (2024-03-23T09:18:53Z) - GraphGLOW: Universal and Generalizable Structure Learning for Graph
Neural Networks [72.01829954658889]
This paper introduces the mathematical definition of this novel problem setting.
We devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs.
The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning.
arXiv Detail & Related papers (2023-06-20T03:33:22Z) - Word Grounded Graph Convolutional Network [24.6338889954789]
Graph Convolutional Networks (GCNs) have shown strong performance in learning text representations for various tasks such as text classification.
We propose to transform the document graph into a word graph, to decouple data samples and a GCN model by using a document-independent graph.
The proposed Word-level Graph (WGraph) can not only implicitly learning word presentation with commonly-used word co-occurrences in corpora, but also incorporate extra global semantic dependency.
arXiv Detail & Related papers (2023-05-10T19:56:55Z) - SHGNN: Structure-Aware Heterogeneous Graph Neural Network [77.78459918119536]
This paper proposes a novel Structure-Aware Heterogeneous Graph Neural Network (SHGNN) to address the above limitations.
We first utilize a feature propagation module to capture the local structure information of intermediate nodes in the meta-path.
Next, we use a tree-attention aggregator to incorporate the graph structure information into the aggregation module on the meta-path.
Finally, we leverage a meta-path aggregator to fuse the information aggregated from different meta-paths.
arXiv Detail & Related papers (2021-12-12T14:18:18Z) - Hierarchical Heterogeneous Graph Representation Learning for Short Text
Classification [60.233529926965836]
We propose a new method called SHINE, which is based on graph neural network (GNN) for short text classification.
First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs.
Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts.
arXiv Detail & Related papers (2021-10-30T05:33:05Z) - GraphFormers: GNN-nested Transformers for Representation Learning on
Textual Graph [53.70520466556453]
We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models.
With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow.
In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
arXiv Detail & Related papers (2021-05-06T12:20:41Z) - Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole.
We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
arXiv Detail & Related papers (2021-01-16T16:18:43Z) - Neural Topic Modeling by Incorporating Document Relationship Graph [18.692100955163713]
Graph Topic Model (GTM) is a GNN based neural topic model that represents a corpus as a document relationship graph.
Documents and words in the corpus become nodes in the graph and are connected based on document-word co-occurrences.
arXiv Detail & Related papers (2020-09-29T12:45:55Z) - Heterogeneous Graph Neural Networks for Extractive Document
Summarization [101.17980994606836]
Cross-sentence relations are a crucial step in extractive document summarization.
We present a graph-based neural network for extractive summarization (HeterSumGraph)
We introduce different types of nodes into graph-based neural networks for extractive document summarization.
arXiv Detail & Related papers (2020-04-26T14:38:11Z) - Every Document Owns Its Structure: Inductive Text Classification via
Graph Neural Networks [22.91359631452695]
We propose TextING for inductive text classification via Graph Neural Networks (GNN)
We first build individual graphs for each document and then use GNN to learn the fine-grained word representations based on their local structures.
Our method outperforms state-of-the-art text classification methods.
arXiv Detail & Related papers (2020-04-22T07:23:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.