BLOSSOM: Block-wise Federated Learning Over Shared and Sparse Observed Modalities

• URL: http://arxiv.org/abs/2603.27552v1
• Date: Sun, 29 Mar 2026 07:25:33 GMT
• Title: BLOSSOM: Block-wise Federated Learning Over Shared and Sparse Observed Modalities
• Abstract summary: Multimodal federated learning (FL) is essential for real-world applications such as autonomous systems and healthcare.<n>We introduce BLOSSOM, a task-agnostic framework for multimodal FL designed to operate under shared and sparsely observed modality conditions.<n>We show that BLOSSOM achieves an average performance gain of 18.7% over full-model aggregation, while in modality-exclusive settings the gain increases to 37.7%.
• Score: 6.503619764155478
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal federated learning (FL) is essential for real-world applications such as autonomous systems and healthcare, where data is distributed across heterogeneous clients with varying and often missing modalities. However, most existing FL approaches assume uniform modality availability, limiting their applicability in practice. We introduce BLOSSOM, a task-agnostic framework for multimodal FL designed to operate under shared and sparsely observed modality conditions. BLOSSOM supports clients with arbitrary modality subsets and enables flexible sharing of model components. To address client and task heterogeneity, we propose a block-wise aggregation strategy that selectively aggregates shared components while keeping task-specific blocks private, enabling partial personalization. We evaluate BLOSSOM on multiple diverse multimodal datasets and analyse the effects of missing modalities and personalization. Our results show that block-wise personalization significantly improves performance, particularly in settings with severe modality sparsity. In modality-incomplete scenarios, BLOSSOM achieves an average performance gain of 18.7% over full-model aggregation, while in modality-exclusive settings the gain increases to 37.7%, highlighting the importance of block-wise learning for practical multimodal FL systems.

Related papers

• UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity [82.84279796262818]
  We propose UniFed-VLM, a unified instruction tuning framework for Vision-Language Models (VLMs)<n>It consists of two key components: 1) Federated Compensated Subspace Aggregation (FedCSA), which performs subspace-aligned aggregation of parameter-efficient adapters.<n>We conduct experiments on multiple benchmark datasets, and the results show that UniFed-VLM achieves stronger average performance.
  arXiv  Detail & Related papers  (2026-08-16T04:01:04Z)
• FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities [7.015345209610299]
  FedTaste is a parameter-efficient framework for topology-aware structural transfer in Multimodal Federated Learning.<n>To adapt clients with missing modalities, we introduce Modality-Adaptive Structural Prompts.<n>In this way, FedTaste avoids explicit modality imputation while preserving shared semantic structure across clients.
  arXiv  Detail & Related papers  (2026-07-25T15:11:58Z)
• Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneity [27.791732281031695]
  Large language models (LLMs) increasingly adopt Mixture-of-Experts (MoE) architectures to scale model capacity while reducing computation.<n>Fine-tuning these MoE-based LLMs often requires access to distributed and privacy-sensitive data, making centralized fine-tuning impractical.<n>We propose FedAlign-MoE, a federated aggregation alignment framework that jointly enforces routing consistency and expert semantic alignment.
  arXiv  Detail & Related papers  (2026-03-22T15:07:39Z)
• TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddings [26.532942920392376]
  TSEmbed is a universal multimodal embedding framework that synergizes Mixture-of-Experts (MoE) with Low-Rank Adaptation (LoRA)<n>We introduce Expert-Aware Negative Sampling (EANS), a novel strategy that leverages expert routing distributions as an intrinsic proxy for semantic similarity.
  arXiv  Detail & Related papers  (2026-03-05T03:43:52Z)
• Not All Clients Are Equal: Collaborative Model Personalization on Heterogeneous Multi-Modal Clients [59.52341877720199]
  We propose FedMosaic, a method that enables knowledge sharing across heterogeneous architectures without huge computational cost.<n>To mimic the real-world task diversity, we propose a multi-modal PFL benchmark spanning 40 distinct tasks with distribution shifts over time.<n>The empirical study shows that FedMosaic outperforms the state-of-the-art PFL methods.
  arXiv  Detail & Related papers  (2025-05-20T09:17:07Z)
• FedEPA: Enhancing Personalization and Modality Alignment in Multimodal Federated Learning [9.531634844824596]
  Federated Learning (FL) enables decentralized model training across multiple parties while preserving privacy.<n>Most FL systems assume clients hold only unimodal data, limiting their real-world applicability.<n>We propose FedEPA, a novel FL framework for multimodal learning.
  arXiv  Detail & Related papers  (2025-04-16T12:32:37Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications.<n> Ensuring their alignment with the diverse preferences of individual users has become a critical challenge.<n>We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic Segmentation [63.31007867379312]
  We introduce U3M: An Unbiased Multiscale Modal Fusion Model for Multimodal Semantics. We employ feature fusion at multiple scales to ensure the effective extraction and integration of both global and local features. Experimental results demonstrate that our approach achieves superior performance across multiple datasets.
  arXiv  Detail & Related papers  (2024-05-24T08:58:48Z)
• Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts [54.529880848937104]
  We develop a unified MLLM with the MoE architecture, named Uni-MoE, that can handle a wide array of modalities. Specifically, it features modality-specific encoders with connectors for a unified multimodal representation. We evaluate the instruction-tuned Uni-MoE on a comprehensive set of multimodal datasets.
  arXiv  Detail & Related papers  (2024-05-18T12:16:01Z)
• Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality [28.90486547668949]
  Multimodal Federated Cross Prototype Learning (MFCPL) is a novel approach for MFL under severely missing modalities.<n>MFCPL provides diverse modality knowledge in modality-shared level with the cross-modal regularization and modality-specific level with cross-modal contrastive mechanism.<n>Our approach introduces the cross-modal alignment to provide regularization for modality-specific features, thereby enhancing the overall performance.
  arXiv  Detail & Related papers  (2024-01-25T02:25:23Z)
• Overcome Modal Bias in Multi-modal Federated Learning via Balanced Modality Selection [19.284989473603627]
  We propose a novel Balanced Modality Selection framework for multi-modal learning (MFL) We show that local training with a certain single modality may contribute more to the global model than training with all local modalities. Our experiments on audio-visual, colored-gray, and front-back datasets showcase the superiority of BMSFed over baselines.
  arXiv  Detail & Related papers  (2023-12-31T05:37:27Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.