Causally-guided Regularization of Graph Attention Improves Generalizability

• URL: http://arxiv.org/abs/2210.10946v1
• Date: Thu, 20 Oct 2022 01:29:10 GMT
• Title: Causally-guided Regularization of Graph Attention Improves Generalizability
• Authors: Alexander P. Wu, Thomas Markovich, Bonnie Berger, Nils Hammerla, Rohit Singh
• Abstract summary: We introduce CAR, a general-purpose regularization framework for graph attention networks. Methodname aligns the attention mechanism with the causal effects of active interventions on graph connectivity. For social media network-sized graphs, a CAR-guided graph rewiring approach could allow us to combine the scalability of graph convolutional methods with the higher performance of graph attention.
• Score: 69.09877209676266
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: However, the inferred attentions are vulnerable to spurious correlations and connectivity in the training data, hampering the generalizability of the model. We introduce CAR, a general-purpose regularization framework for graph attention networks. Embodying a causal inference approach, \methodname aligns the attention mechanism with the causal effects of active interventions on graph connectivity in a scalable manner. CAR is compatible with a variety of graph attention architectures, and we show that it systematically improves generalizability on various node classification tasks. Our ablation studies indicate that \methodname hones in on the aspects of graph structure most pertinent to the prediction (e.g., homophily), and does so more effectively than alternative approaches. Finally, we also show that CAR enhances interpretability of attention weights by accentuating node-neighbor relations that point to causal hypotheses. For social media network-sized graphs, a CAR-guided graph rewiring approach could allow us to combine the scalability of graph convolutional methods with the higher performance of graph attention.

Related papers

• Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers [0.0]
  We reformulate the Transformer's attention mechanism as a graph operation. We introduce Sparse GIN-Attention, a fine-tuning approach that employs sparse GINs.
  arXiv  Detail & Related papers  (2025-01-04T22:30:21Z)
• Catch Causal Signals from Edges for Label Imbalance in Graph Classification [2.0316763723596063]
  We leverage edge information to disentangle the causal subgraph from the original graph. Our design leads to improved performance on graph classification tasks with label imbalance issues. We evaluate our approach on real-word datasets PTC, Tox21, and ogbg-molhiv.
  arXiv  Detail & Related papers  (2025-01-03T09:09:58Z)
• Revealing Decurve Flows for Generalized Graph Propagation [108.80758541147418]
  This study addresses the limitations of the traditional analysis of message-passing, central to graph learning, by defining em textbfgeneralized propagation with directed and weighted graphs. We include a preliminary exploration of learned propagation patterns in datasets, a first in the field.
  arXiv  Detail & Related papers  (2024-02-13T14:13:17Z)
• Semantic Interpretation and Validation of Graph Attention-based Explanations for GNN Models [9.260186030255081]
  We propose a methodology for investigating the use of semantic attention to enhance the explainability of Graph Neural Network (GNN)-based models. Our work extends existing attention-based graph explainability methods by analysing the divergence in the attention distributions in relation to semantically sorted feature sets. We apply our methodology on a lidar pointcloud estimation model successfully identifying key semantic classes that contribute to enhanced performance.
  arXiv  Detail & Related papers  (2023-08-08T12:34:32Z)
• How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision [16.86132592140062]
  We propose a self-supervised graph attention network (SuperGAT) for noisy graphs. We exploit two attention forms compatible with a self-supervised task to predict edges. By encoding edges, SuperGAT learns more expressive attention in distinguishing mislinked neighbors.
  arXiv  Detail & Related papers  (2022-04-11T05:45:09Z)
• GraphCoCo: Graph Complementary Contrastive Learning [65.89743197355722]
  Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. This paper proposes an effective graph complementary contrastive learning approach named GraphCoCo to tackle the above issue.
  arXiv  Detail & Related papers  (2022-03-24T02:58:36Z)
• Learning Robust Representation through Graph Adversarial Contrastive Learning [6.332560610460623]
  Existing studies show that node representations generated by graph neural networks (GNNs) are vulnerable to adversarial attacks. We propose a novel Graph Adversarial Contrastive Learning framework (GraphACL) by introducing adversarial augmentations into graph self-supervised learning.
  arXiv  Detail & Related papers  (2022-01-31T07:07:51Z)
• Joint Graph Learning and Matching for Semantic Feature Correspondence [69.71998282148762]
  We propose a joint emphgraph learning and matching network, named GLAM, to explore reliable graph structures for boosting graph matching. The proposed method is evaluated on three popular visual matching benchmarks (Pascal VOC, Willow Object and SPair-71k) It outperforms previous state-of-the-art graph matching methods by significant margins on all benchmarks.
  arXiv  Detail & Related papers  (2021-09-01T08:24:02Z)
• Diversified Multiscale Graph Learning with Graph Self-Correction [55.43696999424127]
  We propose a diversified multiscale graph learning model equipped with two core ingredients. A graph self-correction (GSC) mechanism to generate informative embedded graphs, and a diversity boosting regularizer (DBR) to achieve a comprehensive characterization of the input graph. Experiments on popular graph classification benchmarks show that the proposed GSC mechanism leads to significant improvements over state-of-the-art graph pooling methods.
  arXiv  Detail & Related papers  (2021-03-17T16:22:24Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.