Fisher Information Embedding for Node and Graph Learning
- URL: http://arxiv.org/abs/2305.07580v2
- Date: Tue, 6 Jun 2023 13:20:34 GMT
- Title: Fisher Information Embedding for Node and Graph Learning
- Authors: Dexiong Chen, Paolo Pellizzoni, Karsten Borgwardt
- Abstract summary: We propose a novel attention-based node embedding framework for graphs.
Our framework builds upon a hierarchical kernel for multisets of subgraphs around nodes.
We provide theoretical insights into generalizability and expressivity of our embeddings.
- Score: 5.263910852465186
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Attention-based graph neural networks (GNNs), such as graph attention
networks (GATs), have become popular neural architectures for processing
graph-structured data and learning node embeddings. Despite their empirical
success, these models rely on labeled data and the theoretical properties of
these models have yet to be fully understood. In this work, we propose a novel
attention-based node embedding framework for graphs. Our framework builds upon
a hierarchical kernel for multisets of subgraphs around nodes (e.g.
neighborhoods) and each kernel leverages the geometry of a smooth statistical
manifold to compare pairs of multisets, by "projecting" the multisets onto the
manifold. By explicitly computing node embeddings with a manifold of Gaussian
mixtures, our method leads to a new attention mechanism for neighborhood
aggregation. We provide theoretical insights into generalizability and
expressivity of our embeddings, contributing to a deeper understanding of
attention-based GNNs. We propose both efficient unsupervised and supervised
methods for learning the embeddings. Through experiments on several node
classification benchmarks, we demonstrate that our proposed method outperforms
existing attention-based graph models like GATs. Our code is available at
https://github.com/BorgwardtLab/fisher_information_embedding.
Related papers
- GNN-LoFI: a Novel Graph Neural Network through Localized Feature-based
Histogram Intersection [51.608147732998994]
Graph neural networks are increasingly becoming the framework of choice for graph-based machine learning.
We propose a new graph neural network architecture that substitutes classical message passing with an analysis of the local distribution of node features.
arXiv Detail & Related papers (2024-01-17T13:04:23Z) - Supervised Attention Using Homophily in Graph Neural Networks [26.77596449192451]
We propose a new technique to encourage higher attention scores between nodes that share the same class label.
We evaluate the proposed method on several node classification datasets demonstrating increased performance over standard baseline models.
arXiv Detail & Related papers (2023-07-11T12:43:23Z) - A Robust Stacking Framework for Training Deep Graph Models with
Multifaceted Node Features [61.92791503017341]
Graph Neural Networks (GNNs) with numerical node features and graph structure as inputs have demonstrated superior performance on various supervised learning tasks with graph data.
The best models for such data types in most standard supervised learning settings with IID (non-graph) data are not easily incorporated into a GNN.
Here we propose a robust stacking framework that fuses graph-aware propagation with arbitrary models intended for IID data.
arXiv Detail & Related papers (2022-06-16T22:46:33Z) - SHGNN: Structure-Aware Heterogeneous Graph Neural Network [77.78459918119536]
This paper proposes a novel Structure-Aware Heterogeneous Graph Neural Network (SHGNN) to address the above limitations.
We first utilize a feature propagation module to capture the local structure information of intermediate nodes in the meta-path.
Next, we use a tree-attention aggregator to incorporate the graph structure information into the aggregation module on the meta-path.
Finally, we leverage a meta-path aggregator to fuse the information aggregated from different meta-paths.
arXiv Detail & Related papers (2021-12-12T14:18:18Z) - A Robust and Generalized Framework for Adversarial Graph Embedding [73.37228022428663]
We propose a robust framework for adversarial graph embedding, named AGE.
AGE generates the fake neighbor nodes as the enhanced negative samples from the implicit distribution.
Based on this framework, we propose three models to handle three types of graph data.
arXiv Detail & Related papers (2021-05-22T07:05:48Z) - Node2Seq: Towards Trainable Convolutions in Graph Neural Networks [59.378148590027735]
We propose a graph network layer, known as Node2Seq, to learn node embeddings with explicitly trainable weights for different neighboring nodes.
For a target node, our method sorts its neighboring nodes via attention mechanism and then employs 1D convolutional neural networks (CNNs) to enable explicit weights for information aggregation.
In addition, we propose to incorporate non-local information for feature learning in an adaptive manner based on the attention scores.
arXiv Detail & Related papers (2021-01-06T03:05:37Z) - CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph
Representation Learning [19.432449825536423]
Unsupervised graph representation learning aims to learn low-dimensional node embeddings without supervision.
We present a novel cluster-aware graph neural network (CAGNN) model for unsupervised graph representation learning using self-supervised techniques.
arXiv Detail & Related papers (2020-09-03T13:57:18Z) - Representation Learning of Graphs Using Graph Convolutional Multilayer
Networks Based on Motifs [17.823543937167848]
mGCMN is a novel framework which utilizes node feature information and the higher order local structure of the graph.
It will greatly improve the learning efficiency of the graph neural network and promote a brand-new learning mode establishment.
arXiv Detail & Related papers (2020-07-31T04:18:20Z) - Graph Neural Networks with Composite Kernels [60.81504431653264]
We re-interpret node aggregation from the perspective of kernel weighting.
We present a framework to consider feature similarity in an aggregation scheme.
We propose feature aggregation as the composition of the original neighbor-based kernel and a learnable kernel to encode feature similarities in a feature space.
arXiv Detail & Related papers (2020-05-16T04:44:29Z) - Convolutional Kernel Networks for Graph-Structured Data [37.13712126432493]
We introduce a family of multilayer graph kernels and establish new links between graph convolutional neural networks and kernel methods.
Our approach generalizes convolutional kernel networks to graph-structured data, by representing graphs as a sequence of kernel feature maps.
Our model can also be trained end-to-end on large-scale data, leading to new types of graph convolutional neural networks.
arXiv Detail & Related papers (2020-03-11T09:44:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.