Uncertainty Estimation for Heterophilic Graphs Through the Lens of Information Theory

• URL: http://arxiv.org/abs/2505.22152v1
• Date: Wed, 28 May 2025 09:18:01 GMT
• Title: Uncertainty Estimation for Heterophilic Graphs Through the Lens of Information Theory
• Authors: Dominik Fuchsgruber, Tom Wollschläger, Johannes Bordne, Stephan Günnemann,
• Abstract summary: We develop a suitable analog to data processing inequality to quantify information throughout the model's layers.<n>We empirically confirm this with a simple post-hoc density estimator on the joint node embedding space.
• Score: 43.50142477752392
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While uncertainty estimation for graphs recently gained traction, most methods rely on homophily and deteriorate in heterophilic settings. We address this by analyzing message passing neural networks from an information-theoretic perspective and developing a suitable analog to data processing inequality to quantify information throughout the model's layers. In contrast to non-graph domains, information about the node-level prediction target can increase with model depth if a node's features are semantically different from its neighbors. Therefore, on heterophilic graphs, the latent embeddings of an MPNN each provide different information about the data distribution - different from homophilic settings. This reveals that considering all node representations simultaneously is a key design principle for epistemic uncertainty estimation on graphs beyond homophily. We empirically confirm this with a simple post-hoc density estimator on the joint node embedding space that provides state-of-the-art uncertainty on heterophilic graphs. At the same time, it matches prior work on homophilic graphs without explicitly exploiting homophily through post-processing.

Related papers

• GRAIN: Multi-Granular and Implicit Information Aggregation Graph Neural Network for Heterophilous Graphs [11.458759345322832]
  Granular and Implicit Graph Network (GRAIN) is a novel GNN model specifically designed for heterophilous graphs.<n>GRAIN enhances node embeddings by aggregating multi-view information at various levels and incorporating implicit data from distant, non-neighboring nodes.<n>We also introduce an adaptive graph information aggregator that efficiently combines multi-granularity and implicit data, significantly improving node representation quality.
  arXiv  Detail & Related papers  (2025-04-09T07:36:44Z)
• Non-Homophilic Graph Pre-Training and Prompt Learning [11.996173149569627]
  We propose ProNoG, a novel pre-training and prompt learning framework for non-homophilic graphs.<n>First, we analyze existing graph pre-training methods, providing theoretical insights into the choice of pre-training tasks.<n>Second, recognizing that each node exhibits unique non-homophilic characteristics, we propose a conditional network to characterize the node-specific patterns in downstream tasks.
  arXiv  Detail & Related papers  (2024-08-22T17:57:31Z)
• Leveraging Invariant Principle for Heterophilic Graph Structure Distribution Shifts [42.77503881972965]
  Heterophilic Graph Neural Networks (HGNNs) have shown promising results for semi-supervised learning tasks on graphs.<n>We propose a framework capable of generating invariant node representations through incorporating heterophily information.<n>Our proposed method can achieve guaranteed performance under heterophilic graph structure distribution shifts.
  arXiv  Detail & Related papers  (2024-08-18T14:10:34Z)
• The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challenges [101.83124435649358]
  Homophily principle, ie nodes with the same labels or similar attributes are more likely to be connected. Recent work has identified a non-trivial set of datasets where GNN's performance compared to the NN's is not satisfactory.
  arXiv  Detail & Related papers  (2024-07-12T18:04:32Z)
• Heterophilous Distribution Propagation for Graph Neural Networks [23.897535976924722]
  We propose heterophilous distribution propagation (HDP) for graph neural networks. Instead of aggregating information from all neighborhoods, HDP adaptively separates the neighbors into homophilous and heterphilous parts. We conduct extensive experiments on 9 benchmark datasets with different levels of homophily.
  arXiv  Detail & Related papers  (2024-05-31T06:40:56Z)
• Generation is better than Modification: Combating High Class Homophily Variance in Graph Anomaly Detection [51.11833609431406]
  Homophily distribution differences between different classes are significantly greater than those in homophilic and heterophilic graphs. We introduce a new metric called Class Homophily Variance, which quantitatively describes this phenomenon. To mitigate its impact, we propose a novel GNN model named Homophily Edge Generation Graph Neural Network (HedGe)
  arXiv  Detail & Related papers  (2024-03-15T14:26:53Z)
• Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All? [61.35457647107439]
  Most real-world homophilic and heterophilic graphs are comprised of a mixture of nodes in both homophilic and heterophilic structural patterns. We provide evidence that Graph Neural Networks(GNNs) on node classification typically perform admirably on homophilic nodes. We then propose a rigorous, non-i.i.d PAC-Bayesian generalization bound for GNNs, revealing reasons for the performance disparity.
  arXiv  Detail & Related papers  (2023-06-02T07:46:20Z)
• Graph Neural Convection-Diffusion with Heterophily [32.234690120340964]
  Graph neural networks (GNNs) have shown promising results across various graph learning tasks. But they often assume homophily, which can result in poor performance on heterophilic graphs. We propose a novel GNN that incorporates the principle of heterophily by modeling the flow of information on nodes.
  arXiv  Detail & Related papers  (2023-05-26T09:47:03Z)
• Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications [52.65389473899139]
  Graphs are a powerful tool for representing and analyzing unstructured, non-Euclidean data ubiquitous in the healthcare domain. Recent works have shown that considering relationships between input data samples have a positive regularizing effect for the downstream task. We propose Graph-in-Graph (GiG), a neural network architecture for protein classification and brain imaging applications.
  arXiv  Detail & Related papers  (2022-04-01T10:01:37Z)
• Block-Approximated Exponential Random Graphs [77.4792558024487]
  An important challenge in the field of exponential random graphs (ERGs) is the fitting of non-trivial ERGs on large graphs. We propose an approximative framework to such non-trivial ERGs that result in dyadic independence (i.e., edge independent) distributions. Our methods are scalable to sparse graphs consisting of millions of nodes.
  arXiv  Detail & Related papers  (2020-02-14T11:42:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.