Mixture of Decoupled Message Passing Experts with Entropy Constraint for General Node Classification

• URL: http://arxiv.org/abs/2502.08083v1
• Date: Wed, 12 Feb 2025 03:10:26 GMT
• Title: Mixture of Decoupled Message Passing Experts with Entropy Constraint for General Node Classification
• Authors: Xuanze Chen, Jiajun Zhou, Jinsong Chen, Shanqing Yu, Qi Xuan,
• Abstract summary: We propose a universal node classification framework based on the Mixture-of-Experts (MoE) mechanism. GNNMoE significantly outperforms mainstream GNNs, heterophilous GNNs, and graph transformers in both node classification performance and universality across diverse graph datasets.
• Score: 6.963363358936621
• License:
• Abstract: The varying degrees of homophily and heterophily in real-world graphs persistently constrain the universality of graph neural networks (GNNs) for node classification. Adopting a data-centric perspective, this work reveals an inherent preference of different graphs towards distinct message encoding schemes: homophilous graphs favor local propagation, while heterophilous graphs exhibit preference for flexible combinations of propagation and transformation. To address this, we propose GNNMoE, a universal node classification framework based on the Mixture-of-Experts (MoE) mechanism. The framework first constructs diverse message-passing experts through recombination of fine-grained encoding operators, then designs soft and hard gating layers to allocate the most suitable expert networks for each node's representation learning, thereby enhancing both model expressiveness and adaptability to diverse graphs. Furthermore, considering that soft gating might introduce encoding noise in homophilous scenarios, we introduce an entropy constraint to guide sharpening of soft gates, achieving organic integration of weighted combination and Top-K selection. Extensive experiments demonstrate that GNNMoE significantly outperforms mainstream GNNs, heterophilous GNNs, and graph transformers in both node classification performance and universality across diverse graph datasets.

Related papers

• Universal Inceptive GNNs by Eliminating the Smoothness-generalization Dilemma [14.493433090244078]
  We propose an Inceptive Graph Neural Net-work (IGNN) that replaces the cascade dependency with an inceptive architecture. Our IGNN outperforms 23 baseline methods,demonstrating superior performance on both homophilic and het-erophilic graphs.
  arXiv  Detail & Related papers  (2024-12-13T02:44:47Z)
• Mixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification [4.129489934631072]
  Graph neural networks excel at graph representation learning but struggle with heterophilous data and long-range dependencies. We propose GNNMoE, a universal model architecture for node classification. We show that GNNMoE performs exceptionally well across various types of graph data, effectively alleviating the over-smoothing issue and global noise.
  arXiv  Detail & Related papers  (2024-12-11T08:35:13Z)
• GPatcher: A Simple and Adaptive MLP Model for Alleviating Graph Heterophily [15.93465948768545]
  We demystify the impact of graph heterophily on graph neural networks (GNNs) filters. We propose a simple yet powerful GNN named GPatcher by leveraging the patch-Mixer architectures. Our model demonstrates outstanding performance on node classification compared with popular homophily GNNs and state-of-the-art heterophily GNNs.
  arXiv  Detail & Related papers  (2023-06-25T20:57:35Z)
• DEGREE: Decomposition Based Explanation For Graph Neural Networks [55.38873296761104]
  We propose DEGREE to provide a faithful explanation for GNN predictions. By decomposing the information generation and aggregation mechanism of GNNs, DEGREE allows tracking the contributions of specific components of the input graph to the final prediction. We also design a subgraph level interpretation algorithm to reveal complex interactions between graph nodes that are overlooked by previous methods.
  arXiv  Detail & Related papers  (2023-05-22T10:29:52Z)
• Relation Embedding based Graph Neural Networks for Handling Heterogeneous Graph [58.99478502486377]
  We propose a simple yet efficient framework to make the homogeneous GNNs have adequate ability to handle heterogeneous graphs. Specifically, we propose Relation Embedding based Graph Neural Networks (RE-GNNs), which employ only one parameter per relation to embed the importance of edge type relations and self-loop connections.
  arXiv  Detail & Related papers  (2022-09-23T05:24:18Z)
• A Variational Edge Partition Model for Supervised Graph Representation Learning [51.30365677476971]
  This paper introduces a graph generative process to model how the observed edges are generated by aggregating the node interactions over a set of overlapping node communities. We partition each edge into the summation of multiple community-specific weighted edges and use them to define community-specific GNNs. A variational inference framework is proposed to jointly learn a GNN based inference network that partitions the edges into different communities, these community-specific GNNs, and a GNN based predictor that combines community-specific GNNs for the end classification task.
  arXiv  Detail & Related papers  (2022-02-07T14:37:50Z)
• Explicit Pairwise Factorized Graph Neural Network for Semi-Supervised Node Classification [59.06717774425588]
  We propose the Explicit Pairwise Factorized Graph Neural Network (EPFGNN), which models the whole graph as a partially observed Markov Random Field. It contains explicit pairwise factors to model output-output relations and uses a GNN backbone to model input-output relations. We conduct experiments on various datasets, which shows that our model can effectively improve the performance for semi-supervised node classification on graphs.
  arXiv  Detail & Related papers  (2021-07-27T19:47:53Z)
• Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patterns [19.346133577539394]
  Graph neural networks (GNNs) have achieved tremendous success on multiple graph-based learning tasks. We focus on transforming the input graph into a computation graph which contains both proximity and structural information. We show that adaptively choosing between structure and proximity leads to improved performance under diverse mixing.
  arXiv  Detail & Related papers  (2021-06-11T19:18:34Z)
• A Unified View on Graph Neural Networks as Graph Signal Denoising [49.980783124401555]
  Graph Neural Networks (GNNs) have risen to prominence in learning representations for graph structured data. In this work, we establish mathematically that the aggregation processes in a group of representative GNN models can be regarded as solving a graph denoising problem. We instantiate a novel GNN model, ADA-UGNN, derived from UGNN, to handle graphs with adaptive smoothness across nodes.
  arXiv  Detail & Related papers  (2020-10-05T04:57:18Z)
• Graph Neural Networks with Heterophily [40.23690407583509]
  We propose a novel framework called CPGNN that generalizes GNNs for graphs with either homophily or heterophily. We show that replacing the compatibility matrix in our framework with the identity (which represents pure homophily) reduces to GCN.
  arXiv  Detail & Related papers  (2020-09-28T18:29:36Z)
• Heterogeneous Graph Transformer [49.675064816860505]
  Heterogeneous Graph Transformer (HGT) architecture for modeling Web-scale heterogeneous graphs. To handle dynamic heterogeneous graphs, we introduce the relative temporal encoding technique into HGT. To handle Web-scale graph data, we design the heterogeneous mini-batch graph sampling algorithm---HGSampling---for efficient and scalable training.
  arXiv  Detail & Related papers  (2020-03-03T04:49:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.