Hierarchical Graph Neural Networks
- URL: http://arxiv.org/abs/2105.03388v1
- Date: Fri, 7 May 2021 16:47:18 GMT
- Title: Hierarchical Graph Neural Networks
- Authors: Stanislav Sobolevsky
- Abstract summary: This paper aims to connect the dots between the traditional Neural Network and the Graph Neural Network architectures.
A Hierarchical Graph Neural Network architecture is proposed, supplementing the original input network layer with the hierarchy of auxiliary network layers.
It enables simultaneous learning of the individual node features along with the aggregated network features at variable resolution and uses them to improve the convergence and stability of the individual node feature learning.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Over the recent years, Graph Neural Networks have become increasingly popular
in network analytic and beyond. With that, their architecture noticeable
diverges from the classical multi-layered hierarchical organization of the
traditional neural networks. At the same time, many conventional approaches in
network science efficiently utilize the hierarchical approaches to account for
the hierarchical organization of the networks, and recent works emphasize their
critical importance. This paper aims to connect the dots between the
traditional Neural Network and the Graph Neural Network architectures as well
as the network science approaches, harnessing the power of the hierarchical
network organization. A Hierarchical Graph Neural Network architecture is
proposed, supplementing the original input network layer with the hierarchy of
auxiliary network layers and organizing the computational scheme updating the
node features through both - horizontal network connections within each layer
as well as the vertical connection between the layers. It enables simultaneous
learning of the individual node features along with the aggregated network
features at variable resolution and uses them to improve the convergence and
stability of the individual node feature learning. The proposed Hierarchical
Graph Neural network architecture is successfully evaluated on the network
embedding and modeling as well as network classification, node labeling, and
community tasks and demonstrates increased efficiency in those.
Related papers
- Discovering Message Passing Hierarchies for Mesh-Based Physics Simulation [61.89682310797067]
We introduce DHMP, which learns Dynamic Hierarchies for Message Passing networks through a differentiable node selection method.
Our experiments demonstrate the effectiveness of DHMP, achieving 22.7% improvement on average compared to recent fixed-hierarchy message passing networks.
arXiv Detail & Related papers (2024-10-03T15:18:00Z) - Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural
Networks via Higher-Order Interactions [1.994307489466967]
This work starts with a theoretical framework to reveal the impact of network's width, depth, and graph topology on the over-squashing phenomena in message-passing neural networks.
The work drifts towards, higher-order interactions and multi-relational inductive biases via Topological Neural Networks.
Inspired by Graph Attention Networks, two topological attention networks are proposed: Simplicial and Cell Attention Networks.
arXiv Detail & Related papers (2024-02-10T08:26:06Z) - GNN-LoFI: a Novel Graph Neural Network through Localized Feature-based
Histogram Intersection [51.608147732998994]
Graph neural networks are increasingly becoming the framework of choice for graph-based machine learning.
We propose a new graph neural network architecture that substitutes classical message passing with an analysis of the local distribution of node features.
arXiv Detail & Related papers (2024-01-17T13:04:23Z) - Modularity Optimization as a Training Criterion for Graph Neural
Networks [2.903711704663904]
We investigate the effect on the quality of learned representations by the incorporation of community structure preservation objectives of networks in the graph convolutional model.
Experimental evaluation on two attributed bibilographic networks showed that the incorporation of the community-preserving objective improves semi-supervised node classification accuracy in the sparse label regime.
arXiv Detail & Related papers (2022-06-30T21:32:33Z) - Subspace Clustering Based Analysis of Neural Networks [7.451579925406617]
We learn affinity graphs from the latent structure of a given neural network layer trained over a set of inputs.
We then use tools from Community Detection to quantify structures present in the input.
We analyze the learned affinity graphs of the final convolutional layer of the network and demonstrate how an input's local neighbourhood affects its classification by the network.
arXiv Detail & Related papers (2021-07-02T22:46:40Z) - Firefly Neural Architecture Descent: a General Approach for Growing
Neural Networks [50.684661759340145]
Firefly neural architecture descent is a general framework for progressively and dynamically growing neural networks.
We show that firefly descent can flexibly grow networks both wider and deeper, and can be applied to learn accurate but resource-efficient neural architectures.
In particular, it learns networks that are smaller in size but have higher average accuracy than those learned by the state-of-the-art methods.
arXiv Detail & Related papers (2021-02-17T04:47:18Z) - Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks [78.65792427542672]
Dynamic Graph Network (DG-Net) is a complete directed acyclic graph, where the nodes represent convolutional blocks and the edges represent connection paths.
Instead of using the same path of the network, DG-Net aggregates features dynamically in each node, which allows the network to have more representation ability.
arXiv Detail & Related papers (2020-10-02T16:50:26Z) - Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
We propose a topological perspective to represent a network into a complete graph for analysis.
By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner.
This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
arXiv Detail & Related papers (2020-08-19T04:53:31Z) - Modeling Dynamic Heterogeneous Network for Link Prediction using
Hierarchical Attention with Temporal RNN [16.362525151483084]
We propose a novel dynamic heterogeneous network embedding method, termed as DyHATR.
It uses hierarchical attention to learn heterogeneous information and incorporates recurrent neural networks with temporal attention to capture evolutionary patterns.
We benchmark our method on four real-world datasets for the task of link prediction.
arXiv Detail & Related papers (2020-04-01T17:16:47Z) - Progressive Graph Convolutional Networks for Semi-Supervised Node
Classification [97.14064057840089]
Graph convolutional networks have been successful in addressing graph-based tasks such as semi-supervised node classification.
We propose a method to automatically build compact and task-specific graph convolutional networks.
arXiv Detail & Related papers (2020-03-27T08:32:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.