Language Model-Enhanced Message Passing for Heterophilic Graph Learning

• URL: http://arxiv.org/abs/2505.19762v1
• Date: Mon, 26 May 2025 09:45:16 GMT
• Title: Language Model-Enhanced Message Passing for Heterophilic Graph Learning
• Authors: Wenjun Wang, Dawei Cheng,
• Abstract summary: We propose a novel language model (LM)-enhanced message passing approach for heterophilic graph leaning (LEMP4HG)<n>Specifically, in the context of text-attributed graph, we provide paired node texts for LM to generate their connection analysis, which are encoded and then fused with paired node textual embeddings through a gating mechanism.<n>The synthesized messages are semantically enriched and adaptively balanced with both nodes' information, which mitigates contradictory signals when neighbor aggregation in heterophilic regions.
• Score: 13.001541910098126
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional graph neural networks (GNNs), which rely on homophily-driven message passing, struggle with heterophilic graphs where connected nodes exhibit dissimilar features and different labels. While existing methods address heterophily through graph structure refinement or adaptation of neighbor aggregation functions, they often overlook the semantic potential of node text, rely on suboptimal message representation for propagation and compromise performance on homophilic graphs. To address these limitations, we propose a novel language model (LM)-enhanced message passing approach for heterophilic graph leaning (LEMP4HG). Specifically, in the context of text-attributed graph, we provide paired node texts for LM to generate their connection analysis, which are encoded and then fused with paired node textual embeddings through a gating mechanism. The synthesized messages are semantically enriched and adaptively balanced with both nodes' information, which mitigates contradictory signals when neighbor aggregation in heterophilic regions. Furthermore, we introduce an active learning strategy guided by our heuristic MVRD (Modulated Variation of Reliable Distance), selectively enhancing node pairs suffer most from message passing, reducing the cost of analysis generation and side effects on homophilic regions. Extensive experiments validate that our approach excels on heterophilic graphs and performs robustly on homophilic ones, with a graph convolutional network (GCN) backbone and a practical budget.

Related papers

• Directed Homophily-Aware Graph Neural Network [7.539052660225002]
  We propose Directed Homophily-aware Graph Neural Network (DHGNN), a novel framework that incorporates homophily-aware and direction-sensitive components.<n>DHGNN employs a resettable gating mechanism to adaptively modulate message contributions based on homophily levels and informativeness.<n>Our analysis shows that the gating mechanism captures directional homophily gaps and fluctuating homophily across layers, providing deeper insights into message-passing behavior on complex graph structures.
  arXiv  Detail & Related papers  (2025-05-28T13:41:04Z)
• Multi-Granular Attention based Heterogeneous Hypergraph Neural Network [5.580244361093485]
  Heterogeneous graph neural networks (HeteGNNs) have demonstrated strong abilities to learn node representations.<n>This paper proposes MGA-HHN, a Multi-Granular Attention based Heterogeneous Hypergraph Neural Network for representation learning.
  arXiv  Detail & Related papers  (2025-05-07T11:42:00Z)
• GRAIN: Multi-Granular and Implicit Information Aggregation Graph Neural Network for Heterophilous Graphs [11.458759345322832]
  Granular and Implicit Graph Network (GRAIN) is a novel GNN model specifically designed for heterophilous graphs.<n>GRAIN enhances node embeddings by aggregating multi-view information at various levels and incorporating implicit data from distant, non-neighboring nodes.<n>We also introduce an adaptive graph information aggregator that efficiently combines multi-granularity and implicit data, significantly improving node representation quality.
  arXiv  Detail & Related papers  (2025-04-09T07:36:44Z)
• Exploring the Potential of Large Language Models for Heterophilic Graphs [38.79574338268997]
  We propose a two-stage framework for modeling heterophilic graphs using large language models (LLMs)<n>In the first stage, we fine-tune the LLM to better identify homophilic and heterophilic edges based on the textual content of their nodes.<n>In the second stage, we adaptively manage message propagation in GNNs for different edge types based on node features, structures, and heterophilic or homophilic characteristics.
  arXiv  Detail & Related papers  (2024-08-26T09:29:56Z)
• Learn from Heterophily: Heterophilous Information-enhanced Graph Neural Network [4.078409998614025]
  Heterophily, nodes with different labels tend to be connected based on semantic meanings, Graph Neural Networks (GNNs) often exhibit suboptimal performance. We propose and demonstrate that the valuable semantic information inherent in heterophily can be utilized effectively in graph learning. We propose HiGNN, an innovative approach that constructs an additional new graph structure, that integrates heterophilous information by leveraging node distribution.
  arXiv  Detail & Related papers  (2024-03-26T03:29:42Z)
• Heterophily-Aware Graph Attention Network [42.640057865981156]
  Graph Neural Networks (GNNs) have shown remarkable success in graph representation learning. Existing heterophilic GNNs tend to ignore the modeling of heterophily of each edge, which is also a vital part in tackling the heterophily problem. We propose a novel Heterophily-Aware Graph Attention Network (HA-GAT) by fully exploring and utilizing the local distribution as the underlying heterophily.
  arXiv  Detail & Related papers  (2023-02-07T03:21:55Z)
• Text Enriched Sparse Hyperbolic Graph Convolutional Networks [21.83127488157701]
  Graph Neural Networks (GNNs) and their hyperbolic variants provide a promising approach to encode such networks in a low-dimensional latent space. We propose Text Enriched Sparse Hyperbolic Graph Convolution Network (TESH-GCN) to capture the graph's metapath structures using semantic signals. Our model outperforms the current state-of-the-art approaches by a large margin on the task of link prediction.
  arXiv  Detail & Related papers  (2022-07-06T00:23:35Z)
• Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions [51.597480162777074]
  Graph neural networks (GNNs) rely on the message passing paradigm to propagate node features and build interactions. Recent works point out that different graph learning tasks require different ranges of interactions between nodes. We study two common graph construction methods in scientific domains, i.e., emphK-nearest neighbor (KNN) graphs and emphfully-connected (FC) graphs.
  arXiv  Detail & Related papers  (2022-05-15T11:38:14Z)
• HL-Net: Heterophily Learning Network for Scene Graph Generation [90.2766568914452]
  We propose a novel Heterophily Learning Network (HL-Net) to explore the homophily and heterophily between objects/relationships in scene graphs. HL-Net comprises the following 1) an adaptive reweighting transformer module, which adaptively integrates the information from different layers to exploit both the heterophily and homophily in objects. We conducted extensive experiments on two public datasets: Visual Genome (VG) and Open Images (OI)
  arXiv  Detail & Related papers  (2022-05-03T06:00:29Z)
• Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning [102.9138736545956]
  Heterogeneous graph neural network (HGNN) is a very popular technique for the modeling and analysis of heterogeneous graphs. We develop for the first time a novel and robust heterogeneous graph contrastive learning approach, namely HGCL, which introduces two views on respective guidance of node attributes and graph topologies. In this new approach, we adopt distinct but most suitable attribute and topology fusion mechanisms in the two views, which are conducive to mining relevant information in attributes and topologies separately.
  arXiv  Detail & Related papers  (2022-04-30T12:57:02Z)
• SHGNN: Structure-Aware Heterogeneous Graph Neural Network [77.78459918119536]
  This paper proposes a novel Structure-Aware Heterogeneous Graph Neural Network (SHGNN) to address the above limitations. We first utilize a feature propagation module to capture the local structure information of intermediate nodes in the meta-path. Next, we use a tree-attention aggregator to incorporate the graph structure information into the aggregation module on the meta-path. Finally, we leverage a meta-path aggregator to fuse the information aggregated from different meta-paths.
  arXiv  Detail & Related papers  (2021-12-12T14:18:18Z)
• Uniting Heterogeneity, Inductiveness, and Efficiency for Graph Representation Learning [68.97378785686723]
  graph neural networks (GNNs) have greatly advanced the performance of node representation learning on graphs. A majority class of GNNs are only designed for homogeneous graphs, leading to inferior adaptivity to the more informative heterogeneous graphs. We propose a novel inductive, meta path-free message passing scheme that packs up heterogeneous node features with their associated edges from both low- and high-order neighbor nodes.
  arXiv  Detail & Related papers  (2021-04-04T23:31:39Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.