Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning

• URL: http://arxiv.org/abs/2602.00067v1
• Date: Tue, 20 Jan 2026 13:38:50 GMT
• Title: Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning
• Authors: Yihan Zhang, Ercan E. Kuruoglu,
• Abstract summary: We propose NSG (Node Splitting Graph)-MoE, a multimodal graph learning framework that integrates a node-splitting and graph-rewiring mechanism.<n>It explicitly decomposes each node into modality-specific components and assigns relation-aware experts to process heterogeneous message flows.<n>Experiments on three multimodal benchmarks demonstrate that NSG-MoE consistently surpasses strong baselines.
• Score: 10.65673380743972
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal graphs are gaining increasing attention due to their rich representational power and wide applicability, yet they introduce substantial challenges arising from severe modality confusion. To address this issue, we propose NSG (Node Splitting Graph)-MoE, a multimodal graph learning framework that integrates a node-splitting and graph-rewiring mechanism with a structured Mixture-of-Experts (MoE) architecture. It explicitly decomposes each node into modality-specific components and assigns relation-aware experts to process heterogeneous message flows, thereby preserving structural information and multimodal semantics while mitigating the undesirable mixing effects commonly observed in general-purpose GNNs. Extensive experiments on three multimodal benchmarks demonstrate that NSG-MoE consistently surpasses strong baselines. Despite incorporating MoE -- which is typically computationally heavy -- our method achieves competitive training efficiency. Beyond empirical results, we provide a spectral analysis revealing that NSG performs adaptive filtering over modality-specific subspaces, thus explaining its disentangling behavior. Furthermore, an information-theoretic analysis shows that the architectural constraints imposed by NSG reduces mutual information between data and parameters and improving generalization capability.

Related papers

• MetaHGNIE: Meta-Path Induced Hypergraph Contrastive Learning in Heterogeneous Knowledge Graphs [16.36978652807043]
  We propose MetaHGNIE, a meta-path induced hypergraph contrastive learning framework for disentangling and aligning structural and semantic information.<n>Our results highlight the effectiveness of explicitly modeling higher-order interactions and cross-modal alignment in heterogeneous knowledge graphs.
  arXiv  Detail & Related papers  (2025-12-13T22:21:33Z)
• Decoupling and Damping: Structurally-Regularized Gradient Matching for Multimodal Graph Condensation [3.2987327415317895]
  We propose Structurally-Regularized Gradient Matching (SR-GM), a novel condensation framework tailored for multimodal graphs.<n> SR-GM significantly improves accuracy and accelerates convergence compared to baseline methods.<n>This research provides a scalable methodology for multimodal graph-based learning in resource-constrained environments.
  arXiv  Detail & Related papers  (2025-11-25T11:50:34Z)
• Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information [91.66597637613263]
  transformer-based models leveraging the attention mechanism have demonstrated strong empirical success in capturing complex dependencies within graphs.<n>We introduce a novel information-theoretic metric: the kernel-guided mutual information (KG-MI) based on the $f$-divergence.<n>We prove that, given sequences generated by a $K$-parent DAG, training a single-layer, multi-head transformer via a gradient ascent converges to the global optimum time.
  arXiv  Detail & Related papers  (2025-10-29T14:07:12Z)
• MoRE-GNN: Multi-omics Data Integration with a Heterogeneous Graph Autoencoder [15.89170003903628]
  MoRE-GNN captures biologically meaningful relationships and outperforms existing methods.<n>MoRE-GNN offers an adaptive, scalable and interpretable framework for advancing multi-omics integration.
  arXiv  Detail & Related papers  (2025-10-08T10:48:15Z)
• ScaleGNN: Towards Scalable Graph Neural Networks via Adaptive High-order Neighboring Feature Fusion [73.85920403511706]
  We propose ScaleGNN, a novel framework that adaptively fuses multi-hop node features for scalable and effective graph learning.<n>We show that ScaleGNN consistently outperforms state-of-the-art GNNs in both predictive accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2025-04-22T14:05:11Z)
• Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detection [28.57277614615255]
  In this paper, we propose a Graph Mixture of Experts (Graph-MoE) network for multivariate time series anomaly detection.<n>Our Graph-MoE can be integrated into any GNN-based MTS anomaly detection method in a plug-and-play manner.<n>In addition, the memory-augmented routers are proposed in this paper to capture the correlation temporal information in terms of the global historical features of MTS.
  arXiv  Detail & Related papers  (2024-12-26T07:49:51Z)
• Mixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification [4.129489934631072]
  Graph neural networks excel at graph representation learning but struggle with heterophilous data and long-range dependencies.<n>We propose GNNMoE, a universal model architecture for node classification.<n>We show that GNNMoE performs exceptionally well across various types of graph data, effectively alleviating the over-smoothing issue and global noise.
  arXiv  Detail & Related papers  (2024-12-11T08:35:13Z)
• DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts [70.21017141742763]
  Graph neural networks (GNNs) are gaining popularity for processing graph-structured data. Existing methods generally use a fixed number of GNN layers to generate representations for all graphs. We propose the depth adaptive mixture of expert (DA-MoE) method, which incorporates two main improvements to GNN.
  arXiv  Detail & Related papers  (2024-11-05T11:46:27Z)
• Simple and Efficient Heterogeneous Graph Neural Network [55.56564522532328]
  Heterogeneous graph neural networks (HGNNs) have powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) over homogeneous graphs, especially the attention mechanism and the multi-layer structure. This paper conducts an in-depth and detailed study of these mechanisms and proposes Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN)
  arXiv  Detail & Related papers  (2022-07-06T10:01:46Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Adversarial Graph Disentanglement [47.27978741175575]
  A real-world graph has a complex topological structure, which is often formed by the interaction of different latent factors. We propose an underlinetextbfAdversarial underlinetextbfDisentangled underlinetextbfGraph underlinetextbfConvolutional underlinetextbfNetwork (ADGCN) for disentangled graph representation learning.
  arXiv  Detail & Related papers  (2021-03-12T14:11:36Z)
• Multi-View Graph Neural Networks for Molecular Property Prediction [67.54644592806876]
  We present Multi-View Graph Neural Network (MV-GNN), a multi-view message passing architecture. In MV-GNN, we introduce a shared self-attentive readout component and disagreement loss to stabilize the training process. We further boost the expressive power of MV-GNN by proposing a cross-dependent message passing scheme.
  arXiv  Detail & Related papers  (2020-05-17T04:46:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.