FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularization

• URL: http://arxiv.org/abs/2410.03070v1
• Date: Fri, 4 Oct 2024 01:24:02 GMT
• Title: FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularization
• Authors: Manh Duong Nguyen, Trung Thanh Nguyen, Huy Hieu Pham, Trong Nghia Hoang, Phi Le Nguyen, Thanh Trung Huynh,
• Abstract summary: Federated Learning (FL) is a method for training machine learning models using distributed data sources. This study proposes a novel framework named FedMAC, designed to address multi-modality missing under conditions of partial-modality missing in FL.
• Score: 11.954904313477176
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Federated Learning (FL) is a method for training machine learning models using distributed data sources. It ensures privacy by allowing clients to collaboratively learn a shared global model while storing their data locally. However, a significant challenge arises when dealing with missing modalities in clients' datasets, where certain features or modalities are unavailable or incomplete, leading to heterogeneous data distribution. While previous studies have addressed the issue of complete-modality missing, they fail to tackle partial-modality missing on account of severe heterogeneity among clients at an instance level, where the pattern of missing data can vary significantly from one sample to another. To tackle this challenge, this study proposes a novel framework named FedMAC, designed to address multi-modality missing under conditions of partial-modality missing in FL. Additionally, to avoid trivial aggregation of multi-modal features, we introduce contrastive-based regularization to impose additional constraints on the latent representation space. The experimental results demonstrate the effectiveness of FedMAC across various client configurations with statistical heterogeneity, outperforming baseline methods by up to 26% in severe missing scenarios, highlighting its potential as a solution for the challenge of partially missing modalities in federated systems.

Related papers

• Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models [83.02797560769285]
  Data-Free Meta-Learning (DFML) aims to derive knowledge from a collection of pre-trained models without accessing their original data. Current methods often overlook the heterogeneity among pre-trained models, which leads to performance degradation due to task conflicts. We propose Task Groupings Regularization, a novel approach that benefits from model heterogeneity by grouping and aligning conflicting tasks.
  arXiv  Detail & Related papers  (2024-05-26T13:11:55Z)
• Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality [31.727012729846333]
  Multimodal Federated Cross Prototype Learning (MFCPL) is a novel approach for MFL under severely missing modalities. MFCPL provides diverse modality knowledge in modality-shared level with the cross-modal regularization and modality-specific level with cross-modal contrastive mechanism. Our approach introduces the cross-modal alignment to provide regularization for modality-specific features, thereby enhancing overall performance.
  arXiv  Detail & Related papers  (2024-01-25T02:25:23Z)
• A review on different techniques used to combat the non-IID and heterogeneous nature of data in FL [0.0]
  Federated Learning (FL) is a machine-learning approach enabling collaborative model training across multiple edge devices. The significance of FL is particularly pronounced in industries such as healthcare and finance, where data privacy holds paramount importance. This report delves into the issues arising from non-IID and heterogeneous data and explores current algorithms designed to address these challenges.
  arXiv  Detail & Related papers  (2024-01-01T16:34:00Z)
• Balanced Multi-modal Federated Learning via Cross-Modal Infiltration [19.513099949266156]
  Federated learning (FL) underpins advancements in privacy-preserving distributed computing. We propose a novel Cross-Modal Infiltration Federated Learning (FedCMI) framework.
  arXiv  Detail & Related papers  (2023-12-31T05:50:15Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Ensemble Modeling for Multimodal Visual Action Recognition [50.38638300332429]
  We propose an ensemble modeling approach for multimodal action recognition. We independently train individual modality models using a variant of focal loss tailored to handle the long-tailed distribution of the MECCANO [21] dataset.
  arXiv  Detail & Related papers  (2023-08-10T08:43:20Z)
• FedGen: Generalizable Federated Learning for Sequential Data [8.784435748969806]
  In many real-world distributed settings, spurious correlations exist due to biases and data sampling issues. We present a generalizable federated learning framework called FedGen, which allows clients to identify and distinguish between spurious and invariant features. We show that FedGen results in models that achieve significantly better generalization and can outperform the accuracy of current federated learning approaches by over 24%.
  arXiv  Detail & Related papers  (2022-11-03T15:48:14Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)
• When Relation Networks meet GANs: Relation GANs with Triplet Loss [110.7572918636599]
  Training stability is still a lingering concern of generative adversarial networks (GANs) In this paper, we explore a relation network architecture for the discriminator and design a triplet loss which performs better generalization and stability. Experiments on benchmark datasets show that the proposed relation discriminator and new loss can provide significant improvement on variable vision tasks.
  arXiv  Detail & Related papers  (2020-02-24T11:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.