Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity

• URL: http://arxiv.org/abs/2605.12584v1
• Date: Tue, 12 May 2026 17:30:15 GMT
• Title: Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity
• Abstract summary: Real-world graphs are often isolated due to data-sharing limitations across multiple parties.<n>We propose scFedMPO, which utilizes topology-aware cross-modal generation to recover missing features.<n>Experiments on 3 tasks across 6 datasets demonstrate that FedMPO outperforms baselines.
• Score: 49.44748281840733
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, multimodal graph learning (MGL) has garnered significant attention for integrating diverse modality information and structured context to support various network applications. However, real-world graphs are often isolated due to data-sharing limitations across multiple parties, and their modalities are frequently incomplete. This highlights an urgent need to develop a robust federated approach. However, we find that existing methods remain insufficient. On the one hand, centralized MGL methods that handle missing modalities overlook the knowledge sharing and generalization in federated scenarios. On the other hand, while federated MGL methods have become increasingly mature, they primarily target non-graph data. Based on these technologies, we identify a two-stage pipeline wherein client-side completion reconstructs missing modalities, and server-side aggregation integrates the client-updated parameters of both the modality generator and the backbone models. Although this serves as a general solution, we identify two primary challenges in achieving greater robustness: (1) Topology-Isolated Local Completion: Client-side modality generation struggles to effectively leverage global semantics. (2) Reliability-Imbalanced Global Aggregation: Server-side multi-party collaboration is hindered by client updates with varying modality availability and recovery reliability. To address these challenges, we propose \textsc{FedMPO}, which utilizes topology-aware cross-modal generation to recover missing features using comprehensive graph context, missing-aware expert routing to locally filter out noisy recovered signals, and reliability-aware aggregation to appropriately down-weight unreliable updates. Extensive experiments on 3 tasks across 6 datasets demonstrate that FedMPO outperforms baselines, achieving performance gains of up to 4.10% and 5.65% in high-missing and non-IID settings.

Related papers

• PRISM: Topology-Aware Cross-Modal Imputation for Modality-Deficient Federated Graph Learning [54.55726437469762]
  We propose textbfPRISM (textbfProactive textbfRetrieval and textbfImputation via textbfStructural textbfMeta-prompting), a topology-aware cross-modal imputation framework.<n>Experiments show PRISM consistently improves modality-deficient clients, outperforming state-of-the-art baselines by textbf4.48% on average.
  arXiv  Detail & Related papers  (2026-06-08T10:08:52Z)
• Robust Multimodal Recommendation via Graph Retrieval-Enhanced Modality Completion [71.20734649881258]
  Real-world multimodal datasets often suffer from modality incompleteness due to sensor failures, annotation scarcity, or privacy constraints.<n>One effective solution is modality completion, which reconstructs missing features to provide modality-complete graphs for downstream tasks.<n>We propose GRE-MC, a Graph Retrieval-Enhanced Modality Completion framework, to overcome these limitations.
  arXiv  Detail & Related papers  (2026-05-01T13:50:52Z)
• OWLEYE: Zero-Shot Learner for Cross-Domain Graph Data Anomaly Detection [48.77471686671269]
  OWLEYE is a novel framework that learns transferable patterns of normal behavior from multiple graphs.<n>We show that OWLEYE achieves superior performance and generalizability compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-01-27T02:08:18Z)
• FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularization [18.276480518649404]
  Federated Learning (FL) is a method for training machine learning models using distributed data sources.<n>This study proposes a novel framework named FedMAC, designed to address multi-modality missing under conditions of partial-modality missing in FL.
  arXiv  Detail & Related papers  (2024-10-04T01:24:02Z)
• Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modality [41.79433449873368]
  We propose a novel multi-modal federated learning method, Federated Multi-modal contrastiVe training with Pre-trained completion (FedMVP) FedMVP integrates the large-scale pre-trained models to enhance the federated training. We demonstrate that the model achieves superior performance over two real-world image-text classification datasets.
  arXiv  Detail & Related papers  (2024-06-16T19:18:06Z)
• FLASH: Federated Learning Across Simultaneous Heterogeneities [55.0981921695672]
  FLASH(Federated Learning Across Simultaneous Heterogeneities) is a lightweight and flexible client selection algorithm.<n>It outperforms state-of-the-art FL frameworks under extensive sources of Heterogeneities.<n>It achieves substantial and consistent improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:04:39Z)
• Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality [28.90486547668949]
  Multimodal Federated Cross Prototype Learning (MFCPL) is a novel approach for MFL under severely missing modalities.<n>MFCPL provides diverse modality knowledge in modality-shared level with the cross-modal regularization and modality-specific level with cross-modal contrastive mechanism.<n>Our approach introduces the cross-modal alignment to provide regularization for modality-specific features, thereby enhancing the overall performance.
  arXiv  Detail & Related papers  (2024-01-25T02:25:23Z)
• Efficient Multimodal Transformer with Dual-Level Feature Restoration for Robust Multimodal Sentiment Analysis [47.29528724322795]
  Multimodal Sentiment Analysis (MSA) has attracted increasing attention recently. Despite significant progress, there are still two major challenges on the way towards robust MSA. We propose a generic and unified framework to address them, named Efficient Multimodal Transformer with Dual-Level Feature Restoration (EMT-DLFR)
  arXiv  Detail & Related papers  (2022-08-16T08:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.