Curvature-based Pooling within Graph Neural Networks

• URL: http://arxiv.org/abs/2308.16516v1
• Date: Thu, 31 Aug 2023 08:00:08 GMT
• Title: Curvature-based Pooling within Graph Neural Networks
• Authors: Cedric Sanders, Andreas Roth, Thomas Liebig
• Abstract summary: Over-squashing and over-smoothing limit the capabilities of graph neural networks (GNNs) CurvPool exploits the notion of curvature of a graph to adaptively identify structures responsible for both over-smoothing and over-squashing. We compare it to other state-of-the-art pooling approaches and establish its competitiveness in terms of classification accuracy, computational complexity, and flexibility.
• Score: 3.1733862899654643
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Over-squashing and over-smoothing are two critical issues, that limit the capabilities of graph neural networks (GNNs). While over-smoothing eliminates the differences between nodes making them indistinguishable, over-squashing refers to the inability of GNNs to propagate information over long distances, as exponentially many node states are squashed into fixed-size representations. Both phenomena share similar causes, as both are largely induced by the graph topology. To mitigate these problems in graph classification tasks, we propose CurvPool, a novel pooling method. CurvPool exploits the notion of curvature of a graph to adaptively identify structures responsible for both over-smoothing and over-squashing. By clustering nodes based on the Balanced Forman curvature, CurvPool constructs a graph with a more suitable structure, allowing deeper models and the combination of distant information. We compare it to other state-of-the-art pooling approaches and establish its competitiveness in terms of classification accuracy, computational complexity, and flexibility. CurvPool outperforms several comparable methods across all considered tasks. The most consistent results are achieved by pooling densely connected clusters using the sum aggregation, as this allows additional information about the size of each pool.

Related papers

• Graph Pooling via Ricci Flow [1.1126342180866644]
  We introduce a graph pooling operator (ORC-Pool) which utilizes a characterization of the graph's geometry via Ollivier's discrete Ricci curvature and an associated geometric flow. ORC-Pool extends such clustering approaches to attributed graphs, allowing for the integration of geometric coarsening into Graph Neural Networks as a pooling layer.
  arXiv  Detail & Related papers  (2024-07-05T03:26:37Z)
• Learning to Reweight for Graph Neural Network [63.978102332612906]
  Graph Neural Networks (GNNs) show promising results for graph tasks. Existing GNNs' generalization ability will degrade when there exist distribution shifts between testing and training graph data. We propose a novel nonlinear graph decorrelation method, which can substantially improve the out-of-distribution generalization ability.
  arXiv  Detail & Related papers  (2023-12-19T12:25:10Z)
• Careful Selection and Thoughtful Discarding: Graph Explicit Pooling Utilizing Discarded Nodes [53.08068729187698]
  We introduce a novel Graph Explicit Pooling (GrePool) method, which selects nodes by explicitly leveraging the relationships between the nodes and final representation vectors crucial for classification. We conduct comprehensive experiments across 12 widely used datasets to validate our proposed method's effectiveness.
  arXiv  Detail & Related papers  (2023-11-21T14:44:51Z)
• NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification [70.51126383984555]
  We introduce a novel all-pair message passing scheme for efficiently propagating node signals between arbitrary nodes. The efficient computation is enabled by a kernerlized Gumbel-Softmax operator. Experiments demonstrate the promising efficacy of the method in various tasks including node classification on graphs.
  arXiv  Detail & Related papers  (2023-06-14T09:21:15Z)
• Addressing Heterophily in Node Classification with Graph Echo State Networks [11.52174067809364]
  We address the challenges of heterophilic graphs with Graph Echo State Network (GESN) for node classification. GESN is a reservoir computing model for graphs, where node embeddings are computed by an untrained message-passing function. Our experiments show that reservoir models are able to achieve better or comparable accuracy with respect to most fully trained deep models.
  arXiv  Detail & Related papers  (2023-05-14T19:42:31Z)
• Higher-order Clustering and Pooling for Graph Neural Networks [77.47617360812023]
  Graph Neural Networks achieve state-of-the-art performance on a plethora of graph classification tasks. HoscPool is a clustering-based graph pooling operator that captures higher-order information hierarchically. We evaluate HoscPool on graph classification tasks and its clustering component on graphs with ground-truth community structure.
  arXiv  Detail & Related papers  (2022-09-02T09:17:10Z)
• Graph Pooling with Maximum-Weight $k$-Independent Sets [12.251091325930837]
  We introduce a graph coarsening mechanism based on the graph-theoretic concept of maximum-weight $k$-independent sets. We prove theoretical guarantees for distortion bounds on path lengths, as well as the ability to preserve key topological properties in the coarsened graphs.
  arXiv  Detail & Related papers  (2022-08-06T14:12:47Z)
• Explicit Pairwise Factorized Graph Neural Network for Semi-Supervised Node Classification [59.06717774425588]
  We propose the Explicit Pairwise Factorized Graph Neural Network (EPFGNN), which models the whole graph as a partially observed Markov Random Field. It contains explicit pairwise factors to model output-output relations and uses a GNN backbone to model input-output relations. We conduct experiments on various datasets, which shows that our model can effectively improve the performance for semi-supervised node classification on graphs.
  arXiv  Detail & Related papers  (2021-07-27T19:47:53Z)
• Second-Order Pooling for Graph Neural Networks [62.13156203025818]
  We propose to use second-order pooling as graph pooling, which naturally solves the above challenges. We show that direct use of second-order pooling with graph neural networks leads to practical problems. We propose two novel global graph pooling methods based on second-order pooling; namely, bilinear mapping and attentional second-order pooling.
  arXiv  Detail & Related papers  (2020-07-20T20:52:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.