DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing

• URL: http://arxiv.org/abs/2401.12780v1
• Date: Tue, 23 Jan 2024 14:06:08 GMT
• Title: DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing
• Authors: Li Sun, Zhenhao Huang, Hua Wu, Junda Ye, Hao Peng, Zhengtao Yu, Philip S. Yu
• Abstract summary: Graph Structure Learning (GSL) plays an important role in boosting Graph Neural Networks (GNNs) with a refined graph. GSL solutions usually focus on structure refinement with task-specific supervision (i.e., node classification) or overlook the inherent weakness of GNNs themselves. We propose to study self-supervised graph structure-feature co-refinement for effectively alleviating the issue of over-squashing in typical GNNs.
• Score: 72.70197960100677
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Graph Neural Networks (GNNs) have shown great power for learning and mining on graphs, and Graph Structure Learning (GSL) plays an important role in boosting GNNs with a refined graph. In the literature, most GSL solutions either primarily focus on structure refinement with task-specific supervision (i.e., node classification), or overlook the inherent weakness of GNNs themselves (e.g., over-squashing), resulting in suboptimal performance despite sophisticated designs. In light of these limitations, we propose to study self-supervised graph structure-feature co-refinement for effectively alleviating the issue of over-squashing in typical GNNs. In this paper, we take a fundamentally different perspective of the Ricci curvature in Riemannian geometry, in which we encounter the challenges of modeling, utilizing and computing Ricci curvature. To tackle these challenges, we present a self-supervised Riemannian model, DeepRicci. Specifically, we introduce a latent Riemannian space of heterogeneous curvatures to model various Ricci curvatures, and propose a gyrovector feature mapping to utilize Ricci curvature for typical GNNs. Thereafter, we refine node features by geometric contrastive learning among different geometric views, and simultaneously refine graph structure by backward Ricci flow based on a novel formulation of differentiable Ricci curvature. Finally, extensive experiments on public datasets show the superiority of DeepRicci, and the connection between backward Ricci flow and over-squashing. Codes of our work are given in https://github.com/RiemanGraph/.

Related papers

• Generalization of Geometric Graph Neural Networks [84.01980526069075]
  We study the generalization capabilities of geometric graph neural networks (GNNs) We prove a generalization gap between the optimal empirical risk and the optimal statistical risk of this GNN. The most important observation is that the generalization capability can be realized with one large graph instead of being limited to the size of the graph as in previous results.
  arXiv  Detail & Related papers  (2024-09-08T18:55:57Z)
• Tackling Oversmoothing in GNN via Graph Sparsification: A Truss-based Approach [1.4854797901022863]
  We propose a novel and flexible truss-based graph sparsification model that prunes edges from dense regions of the graph. We then utilize our sparsification model in the state-of-the-art baseline GNNs and pooling models, such as GIN, SAGPool, GMT, DiffPool, MinCutPool, HGP-SL, DMonPool, and AdamGNN.
  arXiv  Detail & Related papers  (2024-07-16T17:21:36Z)
• Graph Pooling via Ricci Flow [1.1126342180866644]
  We introduce a graph pooling operator (ORC-Pool) which utilizes a characterization of the graph's geometry via Ollivier's discrete Ricci curvature and an associated geometric flow. ORC-Pool extends such clustering approaches to attributed graphs, allowing for the integration of geometric coarsening into Graph Neural Networks as a pooling layer.
  arXiv  Detail & Related papers  (2024-07-05T03:26:37Z)
• Spectral Greedy Coresets for Graph Neural Networks [61.24300262316091]
  The ubiquity of large-scale graphs in node-classification tasks hinders the real-world applications of Graph Neural Networks (GNNs) This paper studies graph coresets for GNNs and avoids the interdependence issue by selecting ego-graphs based on their spectral embeddings. Our spectral greedy graph coreset (SGGC) scales to graphs with millions of nodes, obviates the need for model pre-training, and applies to low-homophily graphs.
  arXiv  Detail & Related papers  (2024-05-27T17:52:12Z)
• Over-Squashing in Riemannian Graph Neural Networks [1.6317061277457001]
  Most graph neural networks (GNNs) are prone to the phenomenon of over-squashing. Recent works have shown that the topology of the graph has the greatest impact on over-squashing. We explore whether over-squashing can be mitigated through the embedding space of the GNN.
  arXiv  Detail & Related papers  (2023-11-27T15:51:07Z)
• How Curvature Enhance the Adaptation Power of Framelet GCNs [27.831929635701886]
  Graph neural network (GNN) has been demonstrated powerful in modeling graph-structured data. This paper introduces a new approach to enhance GNN by discrete graph Ricci curvature. We show that our curvature-based GNN model outperforms the state-of-the-art baselines in both homophily and heterophily graph datasets.
  arXiv  Detail & Related papers  (2023-07-19T06:05:33Z)
• Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces [74.03252813800334]
  Continual graph learning routinely finds its role in a variety of real-world applications where the graph data with different tasks come sequentially. Existing methods work with the zero-curvature Euclidean space, and largely ignore the fact that curvature varies over the coming graph sequence. To address the aforementioned challenges, we propose to explore a challenging yet practical problem, the self-supervised continual graph learning.
  arXiv  Detail & Related papers  (2022-11-30T15:25:27Z)
• ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network [72.16255675586089]
  We propose an Adaptive Curvature Exploration Hyperbolic Graph NeuralNetwork named ACE-HGNN to adaptively learn the optimal curvature according to the input graph and downstream tasks. Experiments on multiple real-world graph datasets demonstrate a significant and consistent performance improvement in model quality with competitive performance and good generalization ability.
  arXiv  Detail & Related papers  (2021-10-15T07:18:57Z)
• Curvature Graph Neural Network [8.477559786537919]
  We introduce discrete graph curvature (the Ricci curvature) to quantify the strength of structural connection of pairwise nodes. We propose Curvature Graph Neural Network (CGNN), which effectively improves the adaptive locality ability of GNNs. The experimental results on synthetic datasets show that CGNN effectively exploits the topology structure information.
  arXiv  Detail & Related papers  (2021-06-30T00:56:03Z)
• Eigen-GNN: A Graph Structure Preserving Plug-in for GNNs [95.63153473559865]
  Graph Neural Networks (GNNs) are emerging machine learning models on graphs. Most existing GNN models in practice are shallow and essentially feature-centric. We show empirically and analytically that the existing shallow GNNs cannot preserve graph structures well. We propose Eigen-GNN, a plug-in module to boost GNNs ability in preserving graph structures.
  arXiv  Detail & Related papers  (2020-06-08T02:47:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.