DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification

• URL: http://arxiv.org/abs/2505.17660v1
• Date: Fri, 23 May 2025 09:23:41 GMT
• Title: DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification
• Authors: Chenyang Li, Jinsong Chen, John E. Hopcroft, Kun He,
• Abstract summary: We propose DAM-GT, Dual positional encoding-based Attention Masking graph Transformer.<n>We show that DAM-GT consistently outperforms state-of-the-art methods in node classification tasks.
• Score: 16.82685813961535
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neighborhood-aware tokenized graph Transformers have recently shown great potential for node classification tasks. Despite their effectiveness, our in-depth analysis of neighborhood tokens reveals two critical limitations in the existing paradigm. First, current neighborhood token generation methods fail to adequately capture attribute correlations within a neighborhood. Second, the conventional self-attention mechanism suffers from attention diversion when processing neighborhood tokens, where high-hop neighborhoods receive disproportionate focus, severely disrupting information interactions between the target node and its neighborhood tokens. To address these challenges, we propose DAM-GT, Dual positional encoding-based Attention Masking graph Transformer. DAM-GT introduces a novel dual positional encoding scheme that incorporates attribute-aware encoding via an attribute clustering strategy, effectively preserving node correlations in both topological and attribute spaces. In addition, DAM-GT formulates a new attention mechanism with a simple yet effective masking strategy to guide interactions between target nodes and their neighborhood tokens, overcoming the issue of attention diversion. Extensive experiments on various graphs with different homophily levels as well as different scales demonstrate that DAM-GT consistently outperforms state-of-the-art methods in node classification tasks.

Related papers

• Multi-Granular Attention based Heterogeneous Hypergraph Neural Network [5.580244361093485]
  Heterogeneous graph neural networks (HeteGNNs) have demonstrated strong abilities to learn node representations.<n>This paper proposes MGA-HHN, a Multi-Granular Attention based Heterogeneous Hypergraph Neural Network for representation learning.
  arXiv  Detail & Related papers  (2025-05-07T11:42:00Z)
• Towards Anomaly-Aware Pre-Training and Fine-Tuning for Graph Anomaly Detection [59.042018542376596]
  Graph anomaly detection (GAD) has garnered increasing attention in recent years, yet remains challenging due to two key factors.<n>Anomaly-Aware Pre-Training and Fine-Tuning (APF) is a framework to mitigate the challenges in GAD.<n> Comprehensive experiments on 10 benchmark datasets validate the superior performance of APF in comparison to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-19T09:57:35Z)
• Higher Order Graph Attention Probabilistic Walk Networks [3.481985817302898]
  Message Passing Neural Networks leverage latent relationships embedded in graph structures. Existing methods rely on local information within the $1$-hop neighborhood. We propose the Higher Order Attention (HoGA) module, which assigns weights to variable-length paths sampled based on feature-vector diversity. HoGA represents higher-order relationships as a robust form of self-attention, applicable to any single-hop attention mechanism.
  arXiv  Detail & Related papers  (2024-11-18T20:46:02Z)
• Degree-Conscious Spiking Graph for Cross-Domain Adaptation [9.785526401884155]
  Spiking Graph Networks (SGNs) have demonstrated significant potential in graph classification by emulating brain-inspired neural dynamics.<n>In this paper, we first propose the domain adaptation problem in SGNs, and introduce a novel framework named Degree-Consicious Spiking Graph for Cross-Domain Adaptation.
  arXiv  Detail & Related papers  (2024-10-09T13:45:54Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• Domain-adaptive Message Passing Graph Neural Network [67.35534058138387]
  Cross-network node classification (CNNC) aims to classify nodes in a label-deficient target network by transferring the knowledge from a source network with abundant labels. We propose a domain-adaptive message passing graph neural network (DM-GNN), which integrates graph neural network (GNN) with conditional adversarial domain adaptation.
  arXiv  Detail & Related papers  (2023-08-31T05:26:08Z)
• BOURNE: Bootstrapped Self-supervised Learning Framework for Unified Graph Anomaly Detection [50.26074811655596]
  We propose a novel unified graph anomaly detection framework based on bootstrapped self-supervised learning (named BOURNE) By swapping the context embeddings between nodes and edges, we enable the mutual detection of node and edge anomalies. BOURNE can eliminate the need for negative sampling, thereby enhancing its efficiency in handling large graphs.
  arXiv  Detail & Related papers  (2023-07-28T00:44:57Z)
• Graph Transformer GANs for Graph-Constrained House Generation [223.739067413952]
  We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations. The GTGAN learns effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task.
  arXiv  Detail & Related papers  (2023-03-14T20:35:45Z)
• Heterogeneous Domain Adaptation for IoT Intrusion Detection: A Geometric Graph Alignment Approach [21.7683532972677]
  Data scarcity hinders the usability of data-dependent algorithms when tackling IoT intrusion detection (IID) We utilise the data rich network intrusion detection (NID) domain to facilitate more accurate intrusion detection for IID domains.
  arXiv  Detail & Related papers  (2023-01-24T03:55:14Z)
• Propagation with Adaptive Mask then Training for Node Classification on Attributed Networks [10.732648536892377]
  node classification on attributed networks is a semi-supervised task that is crucial for network analysis. We propose a new method called the itshape propagation with Adaptive Mask then Training (PAMT) The key idea is to integrate the attribute similarity mask into the structure-aware propagation process. In this way, PAMT could preserve the correlation of the attribute of adjacent nodes during the propagation and effectively reduce the influence of structure noise.
  arXiv  Detail & Related papers  (2022-06-21T06:56:58Z)
• Mixed Graph Contrastive Network for Semi-Supervised Node Classification [63.924129159538076]
  We propose a novel graph contrastive learning method, termed Mixed Graph Contrastive Network (MGCN)<n>In our method, we improve the discriminative capability of the latent embeddings by an unperturbed augmentation strategy and a correlation reduction mechanism.<n>By combining the two settings, we extract rich supervision information from both the abundant nodes and the rare yet valuable labeled nodes for discriminative representation learning.
  arXiv  Detail & Related papers  (2022-06-06T14:26:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.