DeepHGCN: Toward Deeper Hyperbolic Graph Convolutional Networks

• URL: http://arxiv.org/abs/2310.02027v5
• Date: Fri, 9 Aug 2024 12:27:41 GMT
• Title: DeepHGCN: Toward Deeper Hyperbolic Graph Convolutional Networks
• Authors: Jiaxu Liu, Xinping Yi, Xiaowei Huang,
• Abstract summary: We propose DeepHGCN, the first deep multi-layer HGCN architecture with dramatically improved computational efficiency and substantially reduced over-smoothing. DeepHGCN features two key innovations: (1) a novel hyperbolic feature transformation layer that enables fast and accurate linear mappings, and (2) techniques such as hyperbolic residual connections and regularization for both weights and features. Extensive experiments demonstrate that DeepHGCN achieves significant improvements in link prediction and node classification tasks compared to both Euclidean and shallow hyperbolic GCN variants.
• Score: 21.605755985700615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperbolic graph convolutional networks (HGCNs) have demonstrated significant potential in extracting information from hierarchical graphs. However, existing HGCNs are limited to shallow architectures due to the computational expense of hyperbolic operations and the issue of over-smoothing as depth increases. Although treatments have been applied to alleviate over-smoothing in GCNs, developing a hyperbolic solution presents distinct challenges since operations must be carefully designed to fit the hyperbolic nature. Addressing these challenges, we propose DeepHGCN, the first deep multi-layer HGCN architecture with dramatically improved computational efficiency and substantially reduced over-smoothing. DeepHGCN features two key innovations: (1) a novel hyperbolic feature transformation layer that enables fast and accurate linear mappings, and (2) techniques such as hyperbolic residual connections and regularization for both weights and features, facilitated by an efficient hyperbolic midpoint method. Extensive experiments demonstrate that DeepHGCN achieves significant improvements in link prediction and node classification tasks compared to both Euclidean and shallow hyperbolic GCN variants.

Related papers

• Residual Hyperbolic Graph Convolution Networks [24.380072966869932]
  Hyperbolic graph convolutional networks (HGCNs) have demonstrated representational capabilities of modeling hierarchical-structured graphs. Over-smoothing may occur as the number of model layers increases, limiting the representation capabilities of most current HGCN models. We propose residual hyperbolic graph convolutional networks (R-HGCNs) to address the over-smoothing problem.
  arXiv  Detail & Related papers  (2024-12-05T02:38:45Z)
• Spiking Graph Neural Network on Riemannian Manifolds [51.15400848660023]
  Graph neural networks (GNNs) have become the dominant solution for learning on graphs. Existing spiking GNNs consider graphs in Euclidean space, ignoring the structural geometry. We present a Manifold-valued Spiking GNN (MSG) MSG achieves superior performance to previous spiking GNNs and energy efficiency to conventional GNNs.
  arXiv  Detail & Related papers  (2024-10-23T15:09:02Z)
• HGCH: A Hyperbolic Graph Convolution Network Model for Heterogeneous Collaborative Graph Recommendation [11.651443951846668]
  We propose an enhanced HGCN-based model for collaborative filtering that integrates diverse side information into a heterogeneous collaborative graph. We evaluate HGCH on four real datasets, and the results show that HGCH achieves competitive results and outperforms leading baselines.
  arXiv  Detail & Related papers  (2023-04-06T09:38:54Z)
• Binary Graph Convolutional Network with Capacity Exploration [58.99478502486377]
  We propose a Binary Graph Convolutional Network (Bi-GCN), which binarizes both the network parameters and input node attributes. Our Bi-GCN can reduce the memory consumption by an average of 31x for both the network parameters and input data, and accelerate the inference speed by an average of 51x.
  arXiv  Detail & Related papers  (2022-10-24T12:05:17Z)
• GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design [27.311994997480745]
  Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art graph learning model. It can be notoriously challenging to inference GCNs over large graph datasets. This paper proposes a GCN algorithm and accelerator Co-Design framework dubbed GCoD which can largely alleviate the aforementioned GCN irregularity.
  arXiv  Detail & Related papers  (2021-12-22T00:30:50Z)
• Multi-scale Graph Convolutional Networks with Self-Attention [2.66512000865131]
  Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data. Over-smoothing phenomenon as a crucial issue of GCNs remains to be solved and investigated. We propose two novel multi-scale GCN frameworks by incorporating self-attention mechanism and multi-scale information into the design of GCNs.
  arXiv  Detail & Related papers  (2021-12-04T04:41:24Z)
• ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network [72.16255675586089]
  We propose an Adaptive Curvature Exploration Hyperbolic Graph NeuralNetwork named ACE-HGNN to adaptively learn the optimal curvature according to the input graph and downstream tasks. Experiments on multiple real-world graph datasets demonstrate a significant and consistent performance improvement in model quality with competitive performance and good generalization ability.
  arXiv  Detail & Related papers  (2021-10-15T07:18:57Z)
• Fully Hyperbolic Neural Networks [63.22521652077353]
  We propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model. We show that our method has better performance for building both shallow and deep networks.
  arXiv  Detail & Related papers  (2021-05-31T03:36:49Z)
• Simple and Deep Graph Convolutional Networks [63.76221532439285]
  Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Despite their success, most of the current GCN models are shallow, due to the em over-smoothing problem. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques.
  arXiv  Detail & Related papers  (2020-07-04T16:18:06Z)
• DeeperGCN: All You Need to Train Deeper GCNs [66.64739331859226]
  Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs.
  arXiv  Detail & Related papers  (2020-06-13T23:00:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.