Attention-based graph neural networks: a survey

• URL: http://arxiv.org/abs/2605.08679v1
• Date: Sat, 09 May 2026 04:33:14 GMT
• Title: Attention-based graph neural networks: a survey
• Abstract summary: We propose a novel two-level taxonomy for attention-based GNNs.<n>We review these attention-based methods following the proposed taxonomy in detail.<n>We share our thoughts on some open issues and future directions of attention-based GNNs.
• Score: 13.676883481845314
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph neural networks (GNNs) aim to learn well-trained representations in a lower-dimension space for downstream tasks while preserving the topological structures. In recent years, attention mechanism, which is brilliant in the fields of natural language processing and computer vision, is introduced to GNNs to adaptively select the discriminative features and automatically filter the noisy information. To the best of our knowledge, due to the fast-paced advances in this domain, a systematic overview of attention-based GNNs is still missing. To fill this gap, this paper aims to provide a comprehensive survey on recent advances in attention-based GNNs. Firstly, we propose a novel two-level taxonomy for attention-based GNNs from the perspective of development history and architectural perspectives. Specifically, the upper level reveals the three developmental stages of attention-based GNNs, including graph recurrent attention networks, graph attention networks, and graph transformers. The lower level focuses on various typical architectures of each stage. Secondly, we review these attention-based methods following the proposed taxonomy in detail and summarize the advantages and disadvantages of various models. A model characteristics table is also provided for a more comprehensive comparison. Thirdly, we share our thoughts on some open issues and future directions of attention-based GNNs. We hope this survey will provide researchers with an up-to-date reference regarding applications of attention-based GNNs. In addition, to cope with the rapid development in this field, we intend to share the relevant latest papers as an open resource at https://github.com/sunxiaobei/awesome-attention-based-gnns.

Related papers

• A Survey on GNN-based Link Prediction: Techniques, Applications, and Challenges [36.51411347767061]
  Graph Neural Networks (GNNs) have emerged as the leading paradigm for link prediction.<n>This paper provides a comprehensive review of GNN-based link prediction from a novel and dedicated GNN perspective.
  arXiv  Detail & Related papers  (2026-04-29T01:34:00Z)
• Spatiotemporal Graph Learning with Direct Volumetric Information Passing and Feature Enhancement [62.91536661584656]
  We propose a dual-module framework, Cell-embedded and Feature-enhanced Graph Neural Network (aka, CeFeGNN) for learning.<n>We embed learnable cell attributions to the common node-edge message passing process, which better captures the spatial dependency of regional features.<n>Experiments on various PDE systems and one real-world dataset demonstrate that CeFeGNN achieves superior performance compared with other baselines.
  arXiv  Detail & Related papers  (2024-09-26T16:22:08Z)
• The Expressive Power of Graph Neural Networks: A Survey [8.652204270723994]
  We conduct a first survey for models for enhancing expressive power under different forms of definition.<n>The models are reviewed based on three categories, i.e., Graph feature enhancement, Graph topology enhancement, and GNNs architecture enhancement.
  arXiv  Detail & Related papers  (2023-08-16T09:12:21Z)
• A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection [98.41798478488101]
  Time series analytics is crucial to unlocking the wealth of information implicit in available data. Recent advancements in graph neural networks (GNNs) have led to a surge in GNN-based approaches for time series analysis. This survey brings together a vast array of knowledge on GNN-based time series research, highlighting foundations, practical applications, and opportunities of graph neural networks for time series analysis.
  arXiv  Detail & Related papers  (2023-07-07T08:05:03Z)
• Towards Deep Attention in Graph Neural Networks: Problems and Remedies [15.36416000750147]
  Graph neural networks (GNNs) learn the representation of graph-structured data, and their expressiveness can be enhanced by inferring node relations for propagation. We investigate some problematic phenomena related to deep graph attention, including vulnerability to over-smoothed features and smooth cumulative attention. Motivated by our findings, we propose AEROGNN, a novel GNN architecture designed for deep graph attention.
  arXiv  Detail & Related papers  (2023-06-04T15:19:44Z)
• A Survey on Explainability of Graph Neural Networks [4.612101932762187]
  Graph neural networks (GNNs) are powerful graph-based deep-learning models. This survey aims to provide a comprehensive overview of the existing explainability techniques for GNNs.
  arXiv  Detail & Related papers  (2023-06-02T23:36:49Z)
• MentorGNN: Deriving Curriculum for Pre-Training GNNs [61.97574489259085]
  We propose an end-to-end model named MentorGNN that aims to supervise the pre-training process of GNNs across graphs. We shed new light on the problem of domain adaption on relational data (i.e., graphs) by deriving a natural and interpretable upper bound on the generalization error of the pre-trained GNNs.
  arXiv  Detail & Related papers  (2022-08-21T15:12:08Z)
• Automatic Relation-aware Graph Network Proliferation [182.30735195376792]
  We propose Automatic Relation-aware Graph Network Proliferation (ARGNP) for efficiently searching GNNs. These operations can extract hierarchical node/relational information and provide anisotropic guidance for message passing on a graph. Experiments on six datasets for four graph learning tasks demonstrate that GNNs produced by our method are superior to the current state-of-the-art hand-crafted and search-based GNNs.
  arXiv  Detail & Related papers  (2022-05-31T10:38:04Z)
• Federated Graph Neural Networks: Overview, Techniques and Challenges [16.62839758251491]
  Graph neural networks (GNNs) have received significant research attention. As societies become increasingly concerned with data privacy, GNNs face the need to adapt to this new normal. This has led to the rapid development of federated graph neural networks (FedGNNs) research in recent years.
  arXiv  Detail & Related papers  (2022-02-15T09:05:35Z)
• Graph Neural Networks for Graphs with Heterophily: A Survey [98.45621222357397]
  We provide a comprehensive review of graph neural networks (GNNs) for heterophilic graphs. Specifically, we propose a systematic taxonomy that essentially governs existing heterophilic GNN models. We discuss the correlation between graph heterophily and various graph research domains, aiming to facilitate the development of more effective GNNs.
  arXiv  Detail & Related papers  (2022-02-14T23:07:47Z)
• Overcoming Catastrophic Forgetting in Graph Neural Networks [50.900153089330175]
  Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks. We propose a novel scheme dedicated to overcoming this problem and hence strengthen continual learning in graph neural networks (GNNs) At the heart of our approach is a generic module, termed as topology-aware weight preserving(TWP)
  arXiv  Detail & Related papers  (2020-12-10T22:30:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.