OmniFM: Toward Modality-Robust and Task-Agnostic Federated Learning for Heterogeneous Medical Imaging

• URL: http://arxiv.org/abs/2603.21660v1
• Date: Mon, 23 Mar 2026 07:38:08 GMT
• Title: OmniFM: Toward Modality-Robust and Task-Agnostic Federated Learning for Heterogeneous Medical Imaging
• Abstract summary: We propose OmniFM, a modality- and task-agnostic FL framework that unifies training across classification, segmentation, super-resolution, visual question answering, and multimodal fusion.<n>Experiments on real-world datasets show that OmniFM consistently surpasses state-of-the-art FL baselines.
• Score: 5.7838287813897225
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) has become a promising paradigm for collaborative medical image analysis, yet existing frameworks remain tightly coupled to task-specific backbones and are fragile under heterogeneous imaging modalities. Such constraints hinder real-world deployment, where institutions vary widely in modality distributions and must support diverse downstream tasks. To address this limitation, we propose OmniFM, a modality- and task-agnostic FL framework that unifies training across classification, segmentation, super-resolution, visual question answering, and multimodal fusion without re-engineering the optimization pipeline. OmniFM builds on a key frequency-domain insight: low-frequency spectral components exhibit strong cross-modality consistency and encode modality-invariant anatomical structures. Accordingly, OmniFM integrates (i) Global Spectral Knowledge Retrieval to inject global frequency priors, (ii) Embedding-wise Cross-Attention Fusion to align representations, and (iii) Prefix-Suffix Spectral Prompting to jointly condition global and personalized cues, together regularized by a Spectral-Proximal Alignment objective that stabilizes aggregation. Experiments on real-world datasets show that OmniFM consistently surpasses state-of-the-art FL baselines across intra- and cross-modality heterogeneity, achieving superior results under both fine-tuning and training-from-scratch setups.

Related papers

• FUSE: Frequency-domain Unification and Spectral Energy Alignment for Multi-modal Object Re-Identification [41.52868809663972]
  We introduce FUSE, a frequency-domain framework that reformulates multi-modal ReID as a two-stage process of spectral disentanglement and energy alignment.<n>The proposed Spectral Decomposition Module adaptively partitions features into low, mid, and high-frequency subspaces.<n>Experiments on RGBNT201, RGBNT100, and MSVR310 show that FUSE achieves 9.1% mAP and 9.5% Rank-1 improvements.
  arXiv  Detail & Related papers  (2026-06-18T10:21:32Z)
• Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation [58.33732779555114]
  We argue that the key bottleneck is a representation-level failure caused by two coupled heterogeneities.<n>We propose Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation (SpectraMB)<n>SpectraMB achieves the best results in most evaluation settings and exhibits improved robustness under noisy interactions.
  arXiv  Detail & Related papers  (2026-06-01T15:58:13Z)
• Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs [84.3271821505699]
  Chain of Modality (CoM) is an agentic framework that transitions multimodal fusion from passive concatenation to dynamic orchestration.<n> operating in either a training-free or a data-efficient SFT setting, CoM achieves robust and consistent generalization across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-04-16T01:21:14Z)
• FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation [57.930067699003764]
  Federated learning (FL) offers a privacy-preserving paradigm for collaborative medical image analysis without sharing raw data.<n>We introduce FL-MedSegBench, the first comprehensive benchmark for federated learning on medical image segmentation.<n>We systematically evaluate eight generic FL (gFL) and five personalized FL (pFL) methods across multiple dimensions.
  arXiv  Detail & Related papers  (2026-03-12T08:28:41Z)
• Federated Modality-specific Encoders and Partially Personalized Fusion Decoder for Multimodal Brain Tumor Segmentation [42.69983276114173]
  This work proposes a new FL framework with federated modality-specific encoders and partially personalized multimodal fusion decoders.<n>FedMEPD is validated on the BraTS 2018 and 2020 multimodal brain tumor segmentation benchmarks.
  arXiv  Detail & Related papers  (2026-03-05T07:25:58Z)
• Fusion of Heterogeneous Pathology Foundation Models for Whole Slide Image Analysis [10.323462166785133]
  Whole slide image (WSI) analysis has emerged as an increasingly essential technique in computational pathology.<n>Recent advances in pathological foundation models (FMs) have demonstrated significant advantages in deriving meaningful patch-level or slide-level feature representations from WSIs.<n>We propose a novel framework for the fusion of heterogeneous pathological FMs, called FuseCPath, yielding a model with a superior ensemble performance.
  arXiv  Detail & Related papers  (2025-10-31T06:59:11Z)
• AdaFusion: Prompt-Guided Inference with Adaptive Fusion of Pathology Foundation Models [49.550545038402184]
  We propose AdaFusion, a novel prompt-guided inference framework.<n>Our method compresses and aligns tile-level features from diverse models.<n>AdaFusion consistently surpasses individual PFMs across both classification and regression tasks.
  arXiv  Detail & Related papers  (2025-08-07T07:09:31Z)
• Learning to Fuse: Modality-Aware Adaptive Scheduling for Robust Multimodal Foundation Models [0.0]
  Modality-Aware Adaptive Fusion Scheduling (MA-AFS) learns to dynamically modulate the contribution of each modality on a per-instance basis.<n>Our work highlights the importance of adaptive fusion and opens a promising direction toward reliable and uncertainty-aware multimodal learning.
  arXiv  Detail & Related papers  (2025-06-15T05:57:45Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis [28.767460351377462]
  In the context of brain imaging analysis, modality incompleteness presents a significant challenge.<n>We propose ClusMFL, a novel MFL framework that leverages feature clustering for cross-institutional brain imaging analysis.<n>ClusMFL achieves state-of-the-art performance compared to various baseline methods across varying levels of modality incompleteness.
  arXiv  Detail & Related papers  (2025-02-14T09:33:59Z)
• Accelerated Multi-Contrast MRI Reconstruction via Frequency and Spatial Mutual Learning [50.74383395813782]
  We propose a novel Frequency and Spatial Mutual Learning Network (FSMNet) to explore global dependencies across different modalities. The proposed FSMNet achieves state-of-the-art performance for the Multi-Contrast MR Reconstruction task with different acceleration factors.
  arXiv  Detail & Related papers  (2024-09-21T12:02:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.