SMGFM: Spectral Multimodal Graph Pretraining for Multimodal-Attributed Graphs

• URL: http://arxiv.org/abs/2606.12867v2
• Date: Tue, 16 Jun 2026 15:30:36 GMT
• Title: SMGFM: Spectral Multimodal Graph Pretraining for Multimodal-Attributed Graphs
• Abstract summary: Multimodal-attributed graphs (MAGs) couple graph topology with node semantics from text, images, and other modalities.<n>We propose SMGFM, a spectral multimodal graph pretraining framework that decomposes each modality-specific node signal into graph-frequency bands.<n>SMGFM achieves state-of-the-art performance across graph-level and modality-level tasks.
• Score: 38.400117494307615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal-attributed graphs (MAGs) couple graph topology with node semantics from text, images, and other modalities. Traditional graph learning contextualizes node semantics by coupling topology with node features. However, this coupling design becomes troublesome in MAGs, where structure-induced and modality-intrinsic semantics may contribute differently to downstream tasks. Structure-induced semantics promote relational consistency through smooth topological variation, whereas modality-intrinsic semantics often encode local, fine-grained distinctions that should not be uniformly smoothed or aligned. Therefore, the key challenge is to identify semantic roles before cross-modal fusion. To this end, we leverage graph-frequency variation as a prior, where low-frequency components capture topology-consistent semantics and high-frequency components preserve modality-specific semantics. Based on this intuition, we propose SMGFM, a spectral multimodal graph pretraining framework that decomposes each modality-specific node signal into graph-frequency bands and assigns band-level semantic roles before cross-modal interaction. Concretely, SMGFM constructs frequency-resolved modality tokens with scalable Chebyshev filters, estimates their coupling reliability through topology-conditioned routing, and performs band-modality interaction before fusion. Its frequency-routed objectives align smooth consensus routes while preserving modality-specific routes, mitigating spatial-domain entanglement and uniform cross-modal alignment. Extensive experiments conducted on the MAG datasets demonstrate that SMGFM achieves state-of-the-art performance across graph-level and modality-level tasks.

Related papers

• Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity [12.526349151730921]
  We propose Federated multimodal graph learning with Topology-aware Cross-modal Routing (FedTCR)<n>FedTCR employs a two-stage paradigm that comprises federated task-agnostic pre-training followed by isolated task-oriented fine-tuning.<n> Experiments across 7 domains demonstrate that FedTCR outperforms state-of-the-art baselines on both graph-centric and modality-centric tasks.
  arXiv  Detail & Related papers  (2026-08-01T12:29:38Z)
• Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs [8.922335341563363]
  MultimodalAttributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images.<n>Existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion.<n>We propose CoMAG, a unified MAG that learns task-adaptive reliable contexts and modality-preserving alignment within them.
  arXiv  Detail & Related papers  (2026-06-12T06:53:46Z)
• Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs [69.56407708794845]
  MultimodalAttributed Graph Learning (MAGL) integrates node attributes with structural topology via graph aggregation.<n>As pretrained encoders evolve into Large Foundation Models (LFMs), the landscape of MAGL fundamentally shifts.<n>Under high-confidence LFM priors, mandatory aggregation introduces topological intrinsic noise that overwhelms discriminative signals.<n>We propose SUPRA (Shared-Unique Prior-Retaining Architecture), a decoupled dual-pathway paradigm.<n> SUPRA processes modality-specific features through topology-agnostics while capturing structural synergy via a lightweight shared GNN, with auxiliary deep supervision counteracting gradient starvation.
  arXiv  Detail & Related papers  (2026-05-23T17:42:25Z)
• GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective [39.44851011793204]
  We propose a structure-driven post-alignment framework that views frozen multimodal embeddings as graph signals.<n>GOMA decouples three key design choices: where messages should flow, how multimodal evidence should propagate, and which smoothing depth should be retained.<n>All experiments follow a transductive MAG retrieval protocol where the graph serves only as unlabeled context and diagonal self-pair edges are removed.
  arXiv  Detail & Related papers  (2026-05-15T08:21:22Z)
• Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Models [59.8742208800447]
  We propose DyGFM, a Dynamic Graph Foundation Model over multiple domains based on decoupled and divergence-conditioned prompting.<n>We show that DyGFM consistently outperforms 12 state-of-the-art baselines on both node classification and link prediction tasks.
  arXiv  Detail & Related papers  (2026-05-13T13:50:03Z)
• STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learning [48.96526409527749]
  Federated graph learning (FGL) enables collaborative training on graph data across multiple clients.<n>The key challenge is that clients from different modality domains may not share a common semantic space.<n>We propose textbfSTAGE, a protocol-first framework for MM-FGL.
  arXiv  Detail & Related papers  (2026-05-12T10:35:43Z)
• TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution [12.027339810975079]
  Task-aware Modality and Topology co-Evolution (TMTE) is a novel MGL framework that jointly and iteratively optimize graph topology and multimodal representations toward the target task.<n>TMTE consistently achieves state-of-the-art performance across 6 graph-centric and modality-centric tasks.
  arXiv  Detail & Related papers  (2026-03-29T14:54:59Z)
• OptiMAG: Structure-Semantic Alignment via Unbalanced Optimal Transport [37.640303159988015]
  Multimodal Attributed Graphs (MAGs) have been widely adopted for modeling complex systems by integrating multi-modal information, such as text and images, on nodes.<n>We identify a discrepancy between the implicit semantic structure induced by different modality embeddings and the explicit graph structure.<n>Since existing methods typically perform message passing over the fixed explicit graph structure, they inadvertently aggregate dissimilar features.<n>We propose OptiMAG, an Unbalanced Optimal Transport-based regularization framework.
  arXiv  Detail & Related papers  (2026-01-30T11:29:03Z)
• Modality as Heterogeneity: Node Splitting and Graph Rewiring for Multimodal Graph Learning [10.65673380743972]
  We propose NSG (Node Splitting Graph)-MoE, a multimodal graph learning framework that integrates a node-splitting and graph-rewiring mechanism.<n>It explicitly decomposes each node into modality-specific components and assigns relation-aware experts to process heterogeneous message flows.<n>Experiments on three multimodal benchmarks demonstrate that NSG-MoE consistently surpasses strong baselines.
  arXiv  Detail & Related papers  (2026-01-20T13:38:50Z)
• Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression [55.51959317490934]
  Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding.<n>We argue that graphs inherently contain rich structural and semantic information, and that their effective exploitation can unlock potential gains in LLMs reasoning performance.<n>We propose Homophily-aware Structural and Semantic Compression for LLMs (HS2C), a framework centered on exploiting graph homophily.
  arXiv  Detail & Related papers  (2026-01-13T03:35:18Z)
• MetaHGNIE: Meta-Path Induced Hypergraph Contrastive Learning in Heterogeneous Knowledge Graphs [16.36978652807043]
  We propose MetaHGNIE, a meta-path induced hypergraph contrastive learning framework for disentangling and aligning structural and semantic information.<n>Our results highlight the effectiveness of explicitly modeling higher-order interactions and cross-modal alignment in heterogeneous knowledge graphs.
  arXiv  Detail & Related papers  (2025-12-13T22:21:33Z)
• SpatioTemporal Focus for Skeleton-based Action Recognition [66.8571926307011]
  Graph convolutional networks (GCNs) are widely adopted in skeleton-based action recognition. We argue that the performance of recent proposed skeleton-based action recognition methods is limited by the following factors. Inspired by the recent attention mechanism, we propose a multi-grain contextual focus module, termed MCF, to capture the action associated relation information.
  arXiv  Detail & Related papers  (2022-03-31T02:45:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.