Communication-Efficient and Robust Multi-Modal Federated Learning via Latent-Space Consensus

• URL: http://arxiv.org/abs/2603.19067v1
• Date: Thu, 19 Mar 2026 15:56:28 GMT
• Title: Communication-Efficient and Robust Multi-Modal Federated Learning via Latent-Space Consensus
• Abstract summary: Federated learning (FL) enables collaborative model training across distributed devices without sharing raw data.<n>We introduce CoMFed, a Communication-Efficient Multi-Modal Federated Learning framework.<n>A latent-space regularizer aligns these representations across clients, improving cross-modal consistency and robustness to outliers.
• Score: 26.0108296956723
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) enables collaborative model training across distributed devices without sharing raw data, but applying FL to multi-modal settings introduces significant challenges. Clients typically possess heterogeneous modalities and model architectures, making it difficult to align feature spaces efficiently while preserving privacy and minimizing communication costs. To address this, we introduce CoMFed, a Communication-Efficient Multi-Modal Federated Learning framework that uses learnable projection matrices to generate compressed latent representations. A latent-space regularizer aligns these representations across clients, improving cross-modal consistency and robustness to outliers. Experiments on human activity recognition benchmarks show that CoMFed achieves competitive accuracy with minimal overhead.

Related papers

• FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities [7.015345209610299]
  FedTaste is a parameter-efficient framework for topology-aware structural transfer in Multimodal Federated Learning.<n>To adapt clients with missing modalities, we introduce Modality-Adaptive Structural Prompts.<n>In this way, FedTaste avoids explicit modality imputation while preserving shared semantic structure across clients.
  arXiv  Detail & Related papers  (2026-07-25T15:11:58Z)
• Boosting Multimodal Federated Learning via Chained Modality Optimization [69.53075328069504]
  Multimodal Federated Learning (MMFL) enables privacy-preserving collaborative learning across decentralized clients with heterogeneous data and modality availability.<n>To address this, we propose FedMChain, a balanced MMFL framework that structures federated multimodal training as a chain of modality-wise phases.<n>This phase-wise design gives each modality a dedicated local optimization window on multimodal clients to mitigate modality competition, and further promotes cross-modal complementarity via an error-compensated regularizer.
  arXiv  Detail & Related papers  (2026-06-01T08:07:09Z)
• A Step Toward Federated Pretraining of Multimodal Large Language Models [87.64508862413565]
  Federated Learning offers a promising solution to unlock distributed resources.<n>Fed-MA is a lightweight pre-training paradigm that freezes the vision encoder and LLM while collaboratively training the cross-modal projector.<n>We propose Fed-CMP, a pioneering framework for federated MLLM pre-training.
  arXiv  Detail & Related papers  (2026-03-25T08:16:23Z)
• Diffusion-Guided Semantic Consistency for Multimodal Heterogeneity [44.06048719126853]
  Federated learning is severely challenged by non-independent and identically distributed (non-IID) client data.<n>We introduce SemanticFL, a novel framework that leverages the rich semantic representations of pre-trained diffusion models.<n>Our results demonstrate that SemanticFL surpasses existing federated learning approaches, achieving accuracy gains of up to 5.49% over FedAvg.
  arXiv  Detail & Related papers  (2026-03-19T04:44:37Z)
• Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks [63.541114376141735]
  Large language models (LLMs) are proliferating rapidly at the edge, delivering intelligent capabilities across diverse application scenarios.<n>However, their practical deployment in collaborative scenarios confronts fundamental challenges: privacy vulnerabilities, communication overhead, and computational bottlenecks.<n>We propose Federated Attention (FedAttn), which integrates the federated paradigm into the self-attention mechanism.
  arXiv  Detail & Related papers  (2025-11-04T15:14:58Z)
• Communication-Efficient and Interoperable Distributed Learning [27.136816067758215]
  Collaborative learning across heterogeneous model architectures presents significant challenges in ensuring interoperability and preserving privacy.<n>We propose a communication-efficient distributed learning framework that supports model heterogeneity and enables modular composition during inference.
  arXiv  Detail & Related papers  (2025-09-26T18:30:36Z)
• Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients [60.22876915395139]
  This paper studies a challenging robust federated learning task with model heterogeneous and data corrupted clients.<n>Data corruption is unavoidable due to factors such as random noise, compression artifacts, or environmental conditions in real-world deployment.<n>We propose a novel Robust Asymmetric Heterogeneous Federated Learning framework to address these issues.
  arXiv  Detail & Related papers  (2025-03-12T09:52:04Z)
• FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learning [9.084674176224109]
  Federated learning (FL) enables multiple clients with distributed data sources to collaboratively train a shared model without compromising data privacy. We introduce Federated Peer-Adaptive Ensemble Learning (FedPAE), a fully decentralized pFL algorithm that supports model heterogeneity and asynchronous learning. Our approach utilizes a peer-to-peer model sharing mechanism and ensemble selection to achieve a more refined balance between local and global information.
  arXiv  Detail & Related papers  (2024-10-17T22:47:19Z)
• Detached and Interactive Multimodal Learning [17.843121072628477]
  This paper introduces DI-MML, a novel detached MML framework designed to learn complementary information across modalities. It addresses competition by separately training each modality encoder with isolated learning objectives. Experiments conducted on audio-visual, flow-image, and front-rear view datasets show the superior performance of our proposed method.
  arXiv  Detail & Related papers  (2024-07-28T15:38:58Z)
• Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modality [41.79433449873368]
  We propose a novel multi-modal federated learning method, Federated Multi-modal contrastiVe training with Pre-trained completion (FedMVP) FedMVP integrates the large-scale pre-trained models to enhance the federated training. We demonstrate that the model achieves superior performance over two real-world image-text classification datasets.
  arXiv  Detail & Related papers  (2024-06-16T19:18:06Z)
• Non-Federated Multi-Task Split Learning for Heterogeneous Sources [17.47679789733922]
  We introduce a new architecture and methodology to perform multi-task learning for heterogeneous data sources efficiently. We show through theoretical analysis that MTSL can achieve fast convergence by tuning the learning rate of the server and clients.
  arXiv  Detail & Related papers  (2024-05-31T19:27:03Z)
• Balancing Similarity and Complementarity for Federated Learning [91.65503655796603]
  Federated Learning (FL) is increasingly important in mobile and IoT systems. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data. We introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation.
  arXiv  Detail & Related papers  (2024-05-16T08:16:19Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.