Content Augmented Graph Neural Networks

• URL: http://arxiv.org/abs/2311.12741v2
• Date: Sat, 7 Sep 2024 08:18:47 GMT
• Title: Content Augmented Graph Neural Networks
• Authors: Fatemeh Gholamzadeh Nasrabadi, AmirHossein Kashani, Pegah Zahedi, Mostafa Haghir Chehreghani,
• Abstract summary: We propose augmenting nodes' embeddings by embeddings generated from their content, at higher GNN layers. We suggest methods such as using an auto-encoder or building a content graph, to generate content embeddings.
• Score: 0.824969449883056
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, graph neural networks (GNNs) have become a popular tool for solving various problems over graphs. In these models, the link structure of the graph is typically exploited and nodes' embeddings are iteratively updated based on adjacent nodes. Nodes' contents are used solely in the form of feature vectors, served as nodes' first-layer embeddings. However, the filters or convolutions, applied during iterations/layers to these initial embeddings lead to their impact diminish and contribute insignificantly to the final embeddings. In order to address this issue, in this paper we propose augmenting nodes' embeddings by embeddings generated from their content, at higher GNN layers. More precisely, we propose models wherein a structural embedding using a GNN and a content embedding are computed for each node. These two are combined using a combination layer to form the embedding of a node at a given layer layer. We suggest methods such as using an auto-encoder or building a content graph, to generate content embeddings. In the end, by conducting experiments over several real-world datasets, we demonstrate the high accuracy and performance of our models.

Related papers

• GraphViz2Vec: A Structure-aware Feature Generation Model to Improve Classification in GNNs [2.0823678201707034]
  GNNs are widely used to solve various tasks including node classification and link prediction. In this paper, we presented a novel feature extraction methodology GraphViz2Vec that can capture structural information of a node's local neighbourhood. These initial embeddings helps existing models achieve state-of-the-art results in various classification tasks.
  arXiv  Detail & Related papers  (2024-01-30T17:11:04Z)
• Deep Manifold Graph Auto-Encoder for Attributed Graph Embedding [51.75091298017941]
  This paper proposes a novel Deep Manifold (Variational) Graph Auto-Encoder (DMVGAE/DMGAE) for attributed graph data. The proposed method surpasses state-of-the-art baseline algorithms by a significant margin on different downstream tasks across popular datasets.
  arXiv  Detail & Related papers  (2024-01-12T17:57:07Z)
• Degree-based stratification of nodes in Graph Neural Networks [66.17149106033126]
  We modify the Graph Neural Network (GNN) architecture so that the weight matrices are learned, separately, for the nodes in each group. This simple-to-implement modification seems to improve performance across datasets and GNN methods.
  arXiv  Detail & Related papers  (2023-12-16T14:09:23Z)
• Collaborative Graph Neural Networks for Attributed Network Embedding [63.39495932900291]
  Graph neural networks (GNNs) have shown prominent performance on attributed network embedding. We propose COllaborative graph Neural Networks--CONN, a tailored GNN architecture for network embedding.
  arXiv  Detail & Related papers  (2023-07-22T04:52:27Z)
• Contrastive Meta-Learning for Few-shot Node Classification [54.36506013228169]
  Few-shot node classification aims to predict labels for nodes on graphs with only limited labeled nodes as references. We create a novel contrastive meta-learning framework on graphs, named COSMIC, with two key designs.
  arXiv  Detail & Related papers  (2023-06-27T02:22:45Z)
• Reducing Over-smoothing in Graph Neural Networks Using Relational Embeddings [0.15619750966454563]
  We propose a new simple, and efficient method to alleviate the effect of the over-smoothing problem in GNNs. Our method can be used in combination with other methods to give the best performance.
  arXiv  Detail & Related papers  (2023-01-07T19:26:04Z)
• Embedding Compression with Hashing for Efficient Representation Learning in Large-Scale Graph [21.564894767364397]
  Graph neural networks (GNNs) are deep learning models designed specifically for graph data. We develop a node embedding compression method where each node is compactly represented with a bit vector instead of a floating-point vector. We show that the proposed node embedding compression method achieves superior performance compared to the alternatives.
  arXiv  Detail & Related papers  (2022-08-11T05:43:39Z)
• A Robust Stacking Framework for Training Deep Graph Models with Multifaceted Node Features [61.92791503017341]
  Graph Neural Networks (GNNs) with numerical node features and graph structure as inputs have demonstrated superior performance on various supervised learning tasks with graph data. The best models for such data types in most standard supervised learning settings with IID (non-graph) data are not easily incorporated into a GNN. Here we propose a robust stacking framework that fuses graph-aware propagation with arbitrary models intended for IID data.
  arXiv  Detail & Related papers  (2022-06-16T22:46:33Z)
• NDGGNET-A Node Independent Gate based Graph Neural Networks [6.155450481110693]
  For nodes with sparse connectivity, it is difficult to obtain enough information through a single GNN layer. In this thesis, we define a novel framework that allows the normal GNN model to accommodate more layers. Experimental results show that our proposed model can effectively increase the model depth and perform well on several datasets.
  arXiv  Detail & Related papers  (2022-05-11T08:51:04Z)
• Position-based Hash Embeddings For Scaling Graph Neural Networks [8.87527266373087]
  Graph Neural Networks (GNNs) compute node representations by taking into account the topology of the node's ego-network and the features of the ego-network's nodes. When the nodes do not have high-quality features, GNNs learn an embedding layer to compute node embeddings and use them as input features. To reduce the memory associated with this embedding layer, hashing-based approaches, commonly used in applications like NLP and recommender systems, can potentially be used. We present approaches that take advantage of the nodes' position in the graph to dramatically reduce the memory required.
  arXiv  Detail & Related papers  (2021-08-31T22:42:25Z)
• CatGCN: Graph Convolutional Networks with Categorical Node Features [99.555850712725]
  CatGCN is tailored for graph learning when the node features are categorical. We train CatGCN in an end-to-end fashion and demonstrate it on semi-supervised node classification.
  arXiv  Detail & Related papers  (2020-09-11T09:25:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.