Prompt-based Personalized Federated Learning for Medical Visual Question Answering

• URL: http://arxiv.org/abs/2402.09677v1
• Date: Thu, 15 Feb 2024 03:09:54 GMT
• Title: Prompt-based Personalized Federated Learning for Medical Visual Question Answering
• Authors: He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama
• Abstract summary: We present a novel prompt-based personalized federated learning (pFL) method to address data heterogeneity and privacy concerns. We regard medical datasets from different organs as clients and use pFL to train personalized transformer-based VQA models for each client.
• Score: 56.002377299811656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel prompt-based personalized federated learning (pFL) method to address data heterogeneity and privacy concerns in traditional medical visual question answering (VQA) methods. Specifically, we regard medical datasets from different organs as clients and use pFL to train personalized transformer-based VQA models for each client. To address the high computational complexity of client-to-client communication in previous pFL methods, we propose a succinct information sharing system by introducing prompts that are small learnable parameters. In addition, the proposed method introduces a reliability parameter to prevent the negative effects of low performance and irrelevant clients. Finally, extensive evaluations on various heterogeneous medical datasets attest to the effectiveness of our proposed method.

Related papers

• Which Client is Reliable?: A Reliable and Personalized Prompt-based Federated Learning for Medical Image Question Answering [51.26412822853409]
  We present a novel personalized federated learning (pFL) method for medical visual question answering (VQA) models. Our method introduces learnable prompts into a Transformer architecture to efficiently train it on diverse medical datasets without massive computational costs.
  arXiv  Detail & Related papers  (2024-10-23T00:31:17Z)
• Client-Level Differential Privacy via Adaptive Intermediary in Federated Medical Imaging [33.494287036763716]
  Trade-off of differential privacy (DP) between privacy protection and performance is still underexplored for real-world medical scenario. We propose to optimize the trade-off under the context of client-level DP, which focuses on privacy during communications. We propose an adaptive intermediary strategy to improve performance without harming privacy.
  arXiv  Detail & Related papers  (2023-07-24T06:12:37Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• HPN: Personalized Federated Hyperparameter Optimization [41.587553874297676]
  We address two challenges of personalized federated hyperparameter optimization (pFedHPO) handling the exponentially increased search space and characterizing each client without compromising its data privacy. We propose learning a textscHypertextscParameter textscNetwork (HPN) fed with client encoding to decide personalized hyperparameters.
  arXiv  Detail & Related papers  (2023-04-11T13:02:06Z)
• SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization [50.01382938451978]
  We introduce a model-agnostic pipeline that employs soft prompts to diminish variance while preserving the advantages of prompt-based summarization. Experimental findings indicate that our method not only bolsters performance but also effectively curbs variance for various language models.
  arXiv  Detail & Related papers  (2023-03-23T04:47:46Z)
• Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMM [62.62684911017472]
  Federated learning (FL) enables devices to jointly train shared models while keeping the training data local for privacy purposes. We introduce a VFL framework with multiple heads (VIM), which takes the separate contribution of each client into account. VIM achieves significantly higher performance and faster convergence compared with the state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-20T23:14:33Z)
• A Differentially Private Probabilistic Framework for Modeling the Variability Across Federated Datasets of Heterogeneous Multi-View Observations [4.511923587827301]
  We show that our framework can be effectively optimized through expectation (EM) over latent master's distribution and clients' parameters. We tested our method on the analysis of multi-modal medical imaging data and clinical scores from distributed clinical datasets of patients affected by Alzheimer's disease.
  arXiv  Detail & Related papers  (2022-04-15T07:20:47Z)
• Inverse Distance Aggregation for Federated Learning with Non-IID Data [48.48922416867067]
  Federated learning (FL) has been a promising approach in the field of medical imaging in recent years. A critical problem in FL, specifically in medical scenarios is to have a more accurate shared model which is robust to noisy and out-of distribution clients. We propose IDA, a novel adaptive weighting approach for clients based on meta-information which handles unbalanced and non-iid data.
  arXiv  Detail & Related papers  (2020-08-17T23:20:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.