Neural Topic Modeling by Incorporating Document Relationship Graph

• URL: http://arxiv.org/abs/2009.13972v1
• Date: Tue, 29 Sep 2020 12:45:55 GMT
• Title: Neural Topic Modeling by Incorporating Document Relationship Graph
• Authors: Deyu Zhou, Xuemeng Hu, Rui Wang
• Abstract summary: Graph Topic Model (GTM) is a GNN based neural topic model that represents a corpus as a document relationship graph. Documents and words in the corpus become nodes in the graph and are connected based on document-word co-occurrences.
• Score: 18.692100955163713
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Neural Networks (GNNs) that capture the relationships between graph nodes via message passing have been a hot research direction in the natural language processing community. In this paper, we propose Graph Topic Model (GTM), a GNN based neural topic model that represents a corpus as a document relationship graph. Documents and words in the corpus become nodes in the graph and are connected based on document-word co-occurrences. By introducing the graph structure, the relationships between documents are established through their shared words and thus the topical representation of a document is enriched by aggregating information from its neighboring nodes using graph convolution. Extensive experiments on three datasets were conducted and the results demonstrate the effectiveness of the proposed approach.

Related papers

• Graph Neural Networks on Discriminative Graphs of Words [19.817473565906777]
  In this work, we explore a new Discriminative Graph of Words Graph Neural Network (DGoW-GNN) approach to classify text. We propose a new model for the graph-based classification of text, which combines a GNN and a sequence model. We evaluate our approach on seven benchmark datasets and find that it is outperformed by several state-of-the-art baseline models.
  arXiv  Detail & Related papers  (2024-10-27T15:14:06Z)
• DGNN: Decoupled Graph Neural Networks with Structural Consistency between Attribute and Graph Embedding Representations [62.04558318166396]
  Graph neural networks (GNNs) demonstrate a robust capability for representation learning on graphs with complex structures. A novel GNNs framework, dubbed Decoupled Graph Neural Networks (DGNN), is introduced to obtain a more comprehensive embedding representation of nodes. Experimental results conducted on several graph benchmark datasets verify DGNN's superiority in node classification task.
  arXiv  Detail & Related papers  (2024-01-28T06:43:13Z)
• Word Grounded Graph Convolutional Network [24.6338889954789]
  Graph Convolutional Networks (GCNs) have shown strong performance in learning text representations for various tasks such as text classification. We propose to transform the document graph into a word graph, to decouple data samples and a GCN model by using a document-independent graph. The proposed Word-level Graph (WGraph) can not only implicitly learning word presentation with commonly-used word co-occurrences in corpora, but also incorporate extra global semantic dependency.
  arXiv  Detail & Related papers  (2023-05-10T19:56:55Z)
• Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions [51.597480162777074]
  Graph neural networks (GNNs) rely on the message passing paradigm to propagate node features and build interactions. Recent works point out that different graph learning tasks require different ranges of interactions between nodes. We study two common graph construction methods in scientific domains, i.e., emphK-nearest neighbor (KNN) graphs and emphfully-connected (FC) graphs.
  arXiv  Detail & Related papers  (2022-05-15T11:38:14Z)
• Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification [2.064612766965483]
  We propose a novel GNN-based sparse structure learning model for inductive document classification. Our model collects a set of trainable edges connecting disjoint words between sentences and employs structure learning to sparsely select edges with dynamic contextual dependencies. Experiments on several real-world datasets demonstrate that the proposed model outperforms most state-of-the-art results.
  arXiv  Detail & Related papers  (2021-12-13T02:36:04Z)
• A Neural Edge-Editing Approach for Document-Level Relation Graph Extraction [9.449257113935461]
  We treat relations in a document as a relation graph among entities. The relation graph is iteratively constructed by editing edges of an initial graph. The way to edit edges is to classify them in a close-first manner.
  arXiv  Detail & Related papers  (2021-06-18T03:46:49Z)
• GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [53.70520466556453]
  We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow. In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
  arXiv  Detail & Related papers  (2021-05-06T12:20:41Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• CopulaGNN: Towards Integrating Representational and Correlational Roles of Graphs in Graph Neural Networks [23.115288017590093]
  We investigate how Graph Neural Network (GNN) models can effectively leverage both types of information. The proposed Copula Graph Neural Network (CopulaGNN) can take a wide range of GNN models as base models.
  arXiv  Detail & Related papers  (2020-10-05T15:20:04Z)
• Heterogeneous Graph Neural Networks for Extractive Document Summarization [101.17980994606836]
  Cross-sentence relations are a crucial step in extractive document summarization. We present a graph-based neural network for extractive summarization (HeterSumGraph) We introduce different types of nodes into graph-based neural networks for extractive document summarization.
  arXiv  Detail & Related papers  (2020-04-26T14:38:11Z)
• Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs [63.12058935995516]
  Recent graph-to-text models generate text from graph-based data using either global or local aggregation. We propose novel neural models which encode an input graph combining both global and local node contexts. Our approaches lead to significant improvements on two graph-to-text datasets.
  arXiv  Detail & Related papers  (2020-01-29T18:24:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.