HeteGCN: Heterogeneous Graph Convolutional Networks for Text Classification

• URL: http://arxiv.org/abs/2008.12842v1
• Date: Wed, 19 Aug 2020 12:24:35 GMT
• Title: HeteGCN: Heterogeneous Graph Convolutional Networks for Text Classification
• Authors: Rahul Ragesh, Sundararajan Sellamanickam, Arun Iyer, Ram Bairi, Vijay Lingam
• Abstract summary: We propose a heterogeneous graph convolutional network (HeteGCN) modeling approach. The main idea is to learn feature embeddings and derive document embeddings using a HeteGCN architecture. In effect, the number of model parameters is reduced significantly, enabling faster training and improving performance in small labeled training set scenario.
• Score: 1.9739269019020032
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of learning efficient and inductive graph convolutional networks for text classification with a large number of examples and features. Existing state-of-the-art graph embedding based methods such as predictive text embedding (PTE) and TextGCN have shortcomings in terms of predictive performance, scalability and inductive capability. To address these limitations, we propose a heterogeneous graph convolutional network (HeteGCN) modeling approach that unites the best aspects of PTE and TextGCN together. The main idea is to learn feature embeddings and derive document embeddings using a HeteGCN architecture with different graphs used across layers. We simplify TextGCN by dissecting into several HeteGCN models which (a) helps to study the usefulness of individual models and (b) offers flexibility in fusing learned embeddings from different models. In effect, the number of model parameters is reduced significantly, enabling faster training and improving performance in small labeled training set scenario. Our detailed experimental studies demonstrate the efficacy of the proposed approach.

Related papers

• Learning How to Propagate Messages in Graph Neural Networks [55.2083896686782]
  This paper studies the problem of learning message propagation strategies for graph neural networks (GNNs) We introduce the optimal propagation steps as latent variables to help find the maximum-likelihood estimation of the GNN parameters. Our proposed framework can effectively learn personalized and interpretable propagate strategies of messages in GNNs.
  arXiv  Detail & Related papers  (2023-10-01T15:09:59Z)
• Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks? [25.898812694174772]
  This work extensively investigates graph representation methods for text classification. We compare different graph construction schemes using a variety of GNN architectures and setups. Two Transformer-based large language models are also included to complement the study.
  arXiv  Detail & Related papers  (2023-05-23T23:31:24Z)
• Improving Subgraph Representation Learning via Multi-View Augmentation [6.907772294522709]
  Subgraph representation learning based on Graph Neural Network (GNN) has broad applications in chemistry and biology. We develop a novel multiview augmentation mechanism to improve subgraph representation learning and thus the accuracy of downstream prediction tasks.
  arXiv  Detail & Related papers  (2022-05-25T20:17:13Z)
• SStaGCN: Simplified stacking based graph convolutional networks [2.556756699768804]
  Graph convolutional network (GCN) is a powerful model studied broadly in various graph structural data learning tasks. We propose a novel GCN called SStaGCN (Simplified stacking based GCN) by utilizing the ideas of stacking and aggregation. We show that SStaGCN can efficiently mitigate the over-smoothing problem of GCN.
  arXiv  Detail & Related papers  (2021-11-16T05:00:08Z)
• Towards Deeper Graph Neural Networks [63.46470695525957]
  Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Several recent studies attribute this performance deterioration to the over-smoothing issue. We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
  arXiv  Detail & Related papers  (2020-07-18T01:11:14Z)
• Simple and Deep Graph Convolutional Networks [63.76221532439285]
  Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Despite their success, most of the current GCN models are shallow, due to the em over-smoothing problem. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques.
  arXiv  Detail & Related papers  (2020-07-04T16:18:06Z)
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training [62.73470368851127]
  Graph representation learning has emerged as a powerful technique for addressing real-world problems. We design Graph Contrastive Coding -- a self-supervised graph neural network pre-training framework. We conduct experiments on three graph learning tasks and ten graph datasets.
  arXiv  Detail & Related papers  (2020-06-17T16:18:35Z)
• Knowledge Embedding Based Graph Convolutional Network [35.35776808660919]
  This paper proposes a novel framework, namely the Knowledge Embedding based Graph Convolutional Network (KE-GCN) KE-GCN combines the power of Graph Convolutional Network (GCN) in graph-based belief propagation and the strengths of advanced knowledge embedding methods. Our theoretical analysis shows that KE-GCN offers an elegant unification of several well-known GCN methods as specific cases.
  arXiv  Detail & Related papers  (2020-06-12T17:12:51Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)
• Cross-GCN: Enhancing Graph Convolutional Network with $k$-Order Feature Interactions [153.6357310444093]
  Graph Convolutional Network (GCN) is an emerging technique that performs learning and reasoning on graph data. We argue that existing designs of GCN forgo modeling cross features, making GCN less effective for tasks or data where cross features are important. We design a new operator named Cross-feature Graph Convolution, which explicitly models the arbitrary-order cross features with complexity linear to feature dimension and order size.
  arXiv  Detail & Related papers  (2020-03-05T13:05:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.