Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework

• URL: http://arxiv.org/abs/2607.15687v1
• Date: Fri, 17 Jul 2026 06:57:41 GMT
• Title: Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework
• Abstract summary: FedGAMMA casts federated multimodal graph foundation learning as a two-stage semantic-structural alignment problem.<n>During pre-training, a shared-private semantic enhancer disentangles cross-modal commonality from modality-specific information.<n>During fine-tuning, FedGAMMA adapts the pretrained encoder through lightweight graph-aware prompts.
• Score: 35.35423006939282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal-attributed graphs (MAGs), whose nodes carry modalities such as images and text alongside topological structure, now pervade applications including social platforms, e-commerce, and biomedical networks, offering richer semantic signals than single-modality graphs. In practice, such graphs are fragmented across privacy-restricted silos owned by different platforms and institutions, so learning a broadly transferable model over them demands collaborative training that never exposes raw data. This places the task at the intersection of multimodal graph learning and federated learning, yet existing methods cover only one side of it. To address the challenges from these two perspectives, we propose FedGAMMA, casting federated multimodal graph foundation learning as a two-stage semantic-structural alignment problem of federated pre-training and prompt-based fine-tuning. During pre-training, a shared-private semantic enhancer disentangles cross-modal commonality from modality-specific information, aligning it through optimal transport, a topology-aware graph fusion module decouples semantic and structural views via semantic residual graphs and dual positional encodings, and a dual-channel affinity-aware aggregation mechanism estimates client similarity from feature and graph centroids without exposing raw data. During fine-tuning, FedGAMMA adapts the pretrained encoder through lightweight graph-aware prompts, a shared prompt pool with controlled exploration, and channel-wise prompt synchronization. Experiments on twelve multimodal graph datasets show FedGAMMA consistently surpassing a broad range of baselines across downstream tasks, with gains of up to 12.96%. FedGAMMA further outperforms competitive baselines accross multi-domain datasets on multiple tasks with up to 5.71% under few-shot learning scenario.

Related papers

• ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning [9.030717765733398]
  Multimodal graph learning requires jointly training over graph structure and heterogeneous node attributes.<n>We propose ReCoG, a new learning paradigm that tightly couples graph structure learning and multimodal representation learning.<n>Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning.
  arXiv  Detail & Related papers  (2026-08-24T04:16:13Z)
• Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity [12.526349151730921]
  We propose Federated multimodal graph learning with Topology-aware Cross-modal Routing (FedTCR)<n>FedTCR employs a two-stage paradigm that comprises federated task-agnostic pre-training followed by isolated task-oriented fine-tuning.<n> Experiments across 7 domains demonstrate that FedTCR outperforms state-of-the-art baselines on both graph-centric and modality-centric tasks.
  arXiv  Detail & Related papers  (2026-08-01T12:29:38Z)
• FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning [47.940785803361415]
  FedLAB is a traceable semantic codebook framework that organizes multimodal graph knowledge into typed codebooks for modality evidence, node semantics, and topology context.<n>Experiments show that FedLAB improves over state-of-the-art baselines by up to 7.53%, while preserving a native semantic trace interface.
  arXiv  Detail & Related papers  (2026-06-30T17:47:39Z)
• STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learning [48.96526409527749]
  Federated graph learning (FGL) enables collaborative training on graph data across multiple clients.<n>The key challenge is that clients from different modality domains may not share a common semantic space.<n>We propose textbfSTAGE, a protocol-first framework for MM-FGL.
  arXiv  Detail & Related papers  (2026-05-12T10:35:43Z)
• Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach [42.970648490410504]
  Multimodal Graph Foundation Models (MGFMs) allow for leveraging the rich multimodal information in Multimodal-Attributed Graphs (MAGs)<n>We propose PLANET, a novel framework employing a Divide-and-Conquer strategy to decouple modality interaction and alignment across distinct granularities.<n>We show that PLANET significantly outperforms state-of-the-art baselines across diverse graph-centric and multimodal generative tasks.
  arXiv  Detail & Related papers  (2026-02-04T01:05:12Z)
• OWLEYE: Zero-Shot Learner for Cross-Domain Graph Data Anomaly Detection [48.77471686671269]
  OWLEYE is a novel framework that learns transferable patterns of normal behavior from multiple graphs.<n>We show that OWLEYE achieves superior performance and generalizability compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-01-27T02:08:18Z)
• OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models [57.94189874119267]
  Multi-Agent Systems (MAS) offer a powerful paradigm for solving complex problems.<n>Current graph learning-based design methodologies often adhere to a "one-for-one" paradigm.<n>We propose OFA-TAD, a one-for-all framework that generates adaptive collaboration graphs for any task described in natural language.
  arXiv  Detail & Related papers  (2026-01-19T12:23:44Z)
• Graph4MM: Weaving Multimodal Learning with Structural Information [52.16646463590474]
  Graphs provide powerful structural information for modeling intra- and inter-modal relationships.<n>Previous works fail to distinguish multi-hop neighbors and treat the graph as a standalone modality.<n>We propose Graph4MM, a graph-based multimodal learning framework.
  arXiv  Detail & Related papers  (2025-10-19T20:13:03Z)
• Disentangling Homophily and Heterophily in Multimodal Graph Clustering [7.565710850295745]
  Multimodal graphs integrate unstructured heterogeneous data with structured interconnections.<n>Disentangled Multimodal Graph Clustering (DMGC) decomposes hybrid graphs into two complementary views.<n>DMGC achieves state-of-the-art performance, highlighting its effectiveness and generalizability across diverse settings.
  arXiv  Detail & Related papers  (2025-07-21T05:29:53Z)
• UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphs [34.48393396390799]
  We propose a novel cross-domain graph foundation model that enables general representation learning on multimodal graphs.<n>UniGraph2 employs modality-specific encoders alongside a graph neural network (GNN) to learn a unified low-dimensional embedding space.<n>We show that UniGraph2 significantly outperforms state-of-the-art models in tasks such as representation learning, transfer learning, and multimodal generative tasks.
  arXiv  Detail & Related papers  (2025-02-02T14:04:53Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.