Empowering Federated Learning for Massive Models with NVIDIA FLARE

• URL: http://arxiv.org/abs/2402.07792v1
• Date: Mon, 12 Feb 2024 16:59:05 GMT
• Title: Empowering Federated Learning for Massive Models with NVIDIA FLARE
• Authors: Holger R. Roth, Ziyue Xu, Yuan-Ting Hsieh, Adithya Renduchintala, Isaac Yang, Zhihong Zhang, Yuhong Wen, Sean Yang, Kevin Lu, Kristopher Kersten, Camir Ricketts, Daguang Xu, Chester Chen, Yan Cheng, Andrew Feng
• Abstract summary: handling and leveraging data effectively has become a critical challenge. Most state-of-the-art machine learning algorithms are data-centric. In this paper, we explore how federated learning enabled by NVIDIA FLARE can address these challenges.
• Score: 15.732926323081077
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the ever-evolving landscape of artificial intelligence (AI) and large language models (LLMs), handling and leveraging data effectively has become a critical challenge. Most state-of-the-art machine learning algorithms are data-centric. However, as the lifeblood of model performance, necessary data cannot always be centralized due to various factors such as privacy, regulation, geopolitics, copyright issues, and the sheer effort required to move vast datasets. In this paper, we explore how federated learning enabled by NVIDIA FLARE can address these challenges with easy and scalable integration capabilities, enabling parameter-efficient and full supervised fine-tuning of LLMs for natural language processing and biopharmaceutical applications to enhance their accuracy and robustness.

Related papers

• Adaptive Few-Shot Learning (AFSL): Tackling Data Scarcity with Stability, Robustness, and Versatility [3.5897534810405403]
  Few-shot learning enables machine learning models to generalize effectively with minimal labeled data. This paper introduces Adaptive Few-Shot Learning, a framework that integrates meta-learning, domain alignment, noise resilience, and multi-modal integration.
  arXiv  Detail & Related papers  (2025-01-23T08:51:49Z)
• Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data [54.934578742209716]
  In real-world NLP applications, Large Language Models (LLMs) offer promising solutions due to their extensive training on vast datasets. LLKD is an adaptive sample selection method that incorporates signals from both the teacher and student. Our comprehensive experiments show that LLKD achieves superior performance across various datasets with higher data efficiency.
  arXiv  Detail & Related papers  (2024-11-12T18:57:59Z)
• Unsupervised Data Validation Methods for Efficient Model Training [0.0]
  State-of-the-art models in natural language processing (NLP), text-to-speech (TTS), speech-to-text (STT) and vision-language models (VLM) rely heavily on large datasets. This research explores key areas such as defining "quality data," developing methods for generating appropriate data and enhancing accessibility to model training.
  arXiv  Detail & Related papers  (2024-10-10T13:00:53Z)
• FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction [47.336599393600046]
  textscFedNE is a novel approach that integrates the textscFedAvg framework with the contrastive NE technique. We conduct comprehensive experiments on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-09-17T19:23:24Z)
• Generative AI like ChatGPT in Blockchain Federated Learning: use cases, opportunities and future [4.497001527881303]
  This research explores potential integrations of generative AI in federated learning. generative adversarial networks (GANs) and variational autoencoders (VAEs) Generating synthetic data helps federated learning address challenges related to limited data availability.
  arXiv  Detail & Related papers  (2024-07-25T19:43:49Z)
• Best Practices and Lessons Learned on Synthetic Data [83.63271573197026]
  The success of AI models relies on the availability of large, diverse, and high-quality datasets. Synthetic data has emerged as a promising solution by generating artificial data that mimics real-world patterns.
  arXiv  Detail & Related papers  (2024-04-11T06:34:17Z)
• The Frontier of Data Erasure: Machine Unlearning for Large Language Models [56.26002631481726]
  Large Language Models (LLMs) are foundational to AI advancements. LLMs pose risks by potentially memorizing and disseminating sensitive, biased, or copyrighted information. Machine unlearning emerges as a cutting-edge solution to mitigate these concerns.
  arXiv  Detail & Related papers  (2024-03-23T09:26:15Z)
• Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
  We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data. Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
  arXiv  Detail & Related papers  (2023-10-21T12:07:04Z)
• Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly [62.473245910234304]
  This paper takes a hardware-centric approach to explore how Large Language Models can be brought to modern edge computing systems. We provide a micro-level hardware benchmark, compare the model FLOP utilization to a state-of-the-art data center GPU, and study the network utilization in realistic conditions.
  arXiv  Detail & Related papers  (2023-10-04T20:27:20Z)
• FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning [16.86560475992975]
  We introduce Attention-based Machine Unlearning using Federated Reinforcement Learning (FRAMU) FRAMU incorporates adaptive learning mechanisms, privacy preservation techniques, and optimization strategies. Our experiments, conducted on both single-modality and multi-modality datasets, revealed that FRAMU significantly outperformed baseline models.
  arXiv  Detail & Related papers  (2023-09-19T03:13:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.