Interpretable and Generalizable Graph Learning via Stochastic Attention
Mechanism
- URL: http://arxiv.org/abs/2201.12987v1
- Date: Mon, 31 Jan 2022 03:59:48 GMT
- Title: Interpretable and Generalizable Graph Learning via Stochastic Attention
Mechanism
- Authors: Siqi Miao, Miaoyuan Liu, Pan Li
- Abstract summary: Interpretable graph learning is in need as many scientific applications depend on learning models to collect insights from graph-structured data.
Previous works mostly focused on using post-hoc approaches to interpret a pre-trained model.
We propose Graph Attention (GSAT), an attention mechanism derived from the information bottleneck principle.
- Score: 6.289180873978089
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Interpretable graph learning is in need as many scientific applications
depend on learning models to collect insights from graph-structured data.
Previous works mostly focused on using post-hoc approaches to interpret a
pre-trained model (graph neural network models in particular). They argue
against inherently interpretable models because good interpretation of these
models is often at the cost of their prediction accuracy. And, the widely used
attention mechanism for inherent interpretation often fails to provide faithful
interpretation in graph learning tasks. In this work, we address both issues by
proposing Graph Stochastic Attention (GSAT), an attention mechanism derived
from the information bottleneck principle. GSAT leverages stochastic attention
to block the information from the task-irrelevant graph components while
learning stochasticity-reduced attention to select the task-relevant subgraphs
for interpretation. GSAT can also apply to fine-tuning and interpreting
pre-trained models via stochastic attention mechanism. Extensive experiments on
eight datasets show that GSAT outperforms the state-of-the-art methods by up to
20%$\uparrow$ in interpretation AUC and 5%$\uparrow$ in prediction accuracy.
Related papers
- Interpretable Prototype-based Graph Information Bottleneck [22.25047783463307]
We propose a novel framework of explainable Graph Neural Networks (GNNs) called interpretable Prototype-based Graph Information Bottleneck (PGIB)
PGIB incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction.
Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability.
arXiv Detail & Related papers (2023-10-30T18:16:19Z) - Towards Self-Interpretable Graph-Level Anomaly Detection [73.1152604947837]
Graph-level anomaly detection (GLAD) aims to identify graphs that exhibit notable dissimilarity compared to the majority in a collection.
We propose a Self-Interpretable Graph aNomaly dETection model ( SIGNET) that detects anomalous graphs as well as generates informative explanations simultaneously.
arXiv Detail & Related papers (2023-10-25T10:10:07Z) - Semantic Interpretation and Validation of Graph Attention-based
Explanations for GNN Models [9.260186030255081]
We propose a methodology for investigating the use of semantic attention to enhance the explainability of Graph Neural Network (GNN)-based models.
Our work extends existing attention-based graph explainability methods by analysing the divergence in the attention distributions in relation to semantically sorted feature sets.
We apply our methodology on a lidar pointcloud estimation model successfully identifying key semantic classes that contribute to enhanced performance.
arXiv Detail & Related papers (2023-08-08T12:34:32Z) - GraphGLOW: Universal and Generalizable Structure Learning for Graph
Neural Networks [72.01829954658889]
This paper introduces the mathematical definition of this novel problem setting.
We devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs.
The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning.
arXiv Detail & Related papers (2023-06-20T03:33:22Z) - Robust Causal Graph Representation Learning against Confounding Effects [21.380907101361643]
We propose Robust Causal Graph Representation Learning (RCGRL) to learn robust graph representations against confounding effects.
RCGRL introduces an active approach to generate instrumental variables under unconditional moment restrictions, which empowers the graph representation learning model to eliminate confounders.
arXiv Detail & Related papers (2022-08-18T01:31:25Z) - Generalizability Analysis of Graph-based Trajectory Predictor with
Vectorized Representation [29.623692599892365]
Trajectory prediction is one of the essential tasks for autonomous vehicles.
Recent progress in machine learning gave birth to a series of advanced trajectory prediction algorithms.
arXiv Detail & Related papers (2022-08-06T20:19:52Z) - Graph Self-supervised Learning with Accurate Discrepancy Learning [64.69095775258164]
We propose a framework that aims to learn the exact discrepancy between the original and the perturbed graphs, coined as Discrepancy-based Self-supervised LeArning (D-SLA)
We validate our method on various graph-related downstream tasks, including molecular property prediction, protein function prediction, and link prediction tasks, on which our model largely outperforms relevant baselines.
arXiv Detail & Related papers (2022-02-07T08:04:59Z) - Learning on Random Balls is Sufficient for Estimating (Some) Graph
Parameters [28.50409304490877]
We develop a theoretical framework for graph classification problems in the partial observation setting.
We propose a new graph classification model that works on a randomly sampled subgraph.
arXiv Detail & Related papers (2021-11-05T08:32:46Z) - Towards Open-World Feature Extrapolation: An Inductive Graph Learning
Approach [80.8446673089281]
We propose a new learning paradigm with graph representation and learning.
Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
arXiv Detail & Related papers (2021-10-09T09:02:45Z) - Interpreting Graph Neural Networks for NLP With Differentiable Edge
Masking [63.49779304362376]
Graph neural networks (GNNs) have become a popular approach to integrating structural inductive biases into NLP models.
We introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges.
We show that we can drop a large proportion of edges without deteriorating the performance of the model.
arXiv Detail & Related papers (2020-10-01T17:51:19Z) - GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training [62.73470368851127]
Graph representation learning has emerged as a powerful technique for addressing real-world problems.
We design Graph Contrastive Coding -- a self-supervised graph neural network pre-training framework.
We conduct experiments on three graph learning tasks and ten graph datasets.
arXiv Detail & Related papers (2020-06-17T16:18:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.