SEMFED: Semantic-Aware Resource-Efficient Federated Learning for Heterogeneous NLP Tasks

• URL: http://arxiv.org/abs/2505.23801v1
• Date: Mon, 26 May 2025 20:58:13 GMT
• Title: SEMFED: Semantic-Aware Resource-Efficient Federated Learning for Heterogeneous NLP Tasks
• Abstract summary: This paper introduces SEMFED, a novel semantic-aware resource-efficient federated learning framework specifically designed for heterogeneous NLP tasks.<n> SEMFED incorporates three key innovations: (1) a semantic-aware client selection mechanism that balances semantic diversity with resource constraints, (2) adaptive NLP-specific model architectures tailored to device capabilities while preserving semantic information, and (3) a communication-efficient semantic feature compression technique that significantly reduces bandwidth requirements.
• Score: 0.5461938536945723
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: Federated Learning (FL) has emerged as a promising paradigm for training machine learning models while preserving data privacy. However, applying FL to Natural Language Processing (NLP) tasks presents unique challenges due to semantic heterogeneity across clients, vocabulary mismatches, and varying resource constraints on edge devices. Objectives: This paper introduces SEMFED, a novel semantic-aware resource-efficient federated learning framework specifically designed for heterogeneous NLP tasks. Methods: SEMFED incorporates three key innovations: (1) a semantic-aware client selection mechanism that balances semantic diversity with resource constraints, (2) adaptive NLP-specific model architectures tailored to device capabilities while preserving semantic information, and (3) a communication-efficient semantic feature compression technique that significantly reduces bandwidth requirements. Results: Experimental results on various NLP classification tasks demonstrate that SEMFED achieves an 80.5% reduction in communication costs while maintaining model accuracy above 98%, outperforming state-of-the-art FL approaches. Conclusion: SEMFED effectively manages heterogeneous client environments with varying computational resources, network reliability, and semantic data distributions, making it particularly suitable for real-world federated NLP deployments.

Related papers

• Diffusion-Guided Semantic Consistency for Multimodal Heterogeneity [44.06048719126853]
  Federated learning is severely challenged by non-independent and identically distributed (non-IID) client data.<n>We introduce SemanticFL, a novel framework that leverages the rich semantic representations of pre-trained diffusion models.<n>Our results demonstrate that SemanticFL surpasses existing federated learning approaches, achieving accuracy gains of up to 5.49% over FedAvg.
  arXiv  Detail & Related papers  (2026-03-19T04:44:37Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• Bridging VLMs and Embodied Intelligence with Deliberate Practice Policy Optimization [72.20212909644017]
  Deliberate Practice Policy Optimization (DPPO) is a metacognitive Metaloop'' training framework.<n>DPPO alternates between supervised fine-tuning (competence expansion) and reinforcement learning (skill refinement)<n> Empirically, training a vision-language embodied model with DPPO, referred to as Pelican-VL 1.0, yields a 20.3% performance improvement over the base model.<n>We are open-sourcing both the models and code, providing the first systematic framework that alleviates the data and resource bottleneck.
  arXiv  Detail & Related papers  (2025-11-20T17:58:04Z)
• Implicit Federated In-context Learning For Task-Specific LLM Fine-Tuning [10.042856500868805]
  We propose the Implicit Federated In-Context Learning (IFed-ICL) framework.<n>IFed-ICL draws inspiration from federated learning to establish a novel distributed collaborative paradigm.<n>Compared to traditional methods, IFed-ICL avoids the extensive parameter updates required by conventional fine-tuning methods.
  arXiv  Detail & Related papers  (2025-11-10T06:34:29Z)
• CoT-Saliency: Unified Chain-of-Thought Reasoning for Heterogeneous Saliency Tasks [96.64597365827046]
  We present the first unified framework that jointly handles three operationally heterogeneous saliency tasks.<n>We introduce a Chain-of-Thought (CoT) reasoning process in a Vision-Language Model (VLM) to bridge task heterogeneity.<n>We show our model matches or outperforms specialized SOTA methods and strong closed-source VLMs across all tasks.
  arXiv  Detail & Related papers  (2025-11-01T04:37:01Z)
• FedHFT: Efficient Federated Finetuning with Heterogeneous Edge Clients [37.60974510385015]
  Fine-tuning pre-trained large language models (LLMs) has become a common practice for personalized natural language understanding (NLU) applications.<n>There are two main challenges: (i) limited and/or heterogeneous data for fine-tuning due to proprietary data confidentiality or privacy requirements, and (ii) varying resources available across participating clients such as edge devices.<n>This paper presents FedHFT - an efficient and personalized federated fine-tuning framework to address both challenges.
  arXiv  Detail & Related papers  (2025-10-15T19:47:51Z)
• FedPref: Federated Learning Across Heterogeneous Multi-objective Preferences [2.519319150166215]
  Federated Learning (FL) is a distributed machine learning strategy developed for settings where training data is owned by distributed devices and cannot be shared.<n>The application of FL to real-world settings brings additional challenges associated with heterogeneity between participants.<n>We propose FedPref, a first algorithm designed to facilitate personalised FL in this setting.
  arXiv  Detail & Related papers  (2025-01-23T12:12:59Z)
• FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models [50.331708897857574]
  We introduce FactorLLM, a novel approach that decomposes well-trained dense FFNs into sparse sub-networks without requiring any further modifications. FactorLLM achieves comparable performance to the source model securing up to 85% model performance while obtaining over a 30% increase in inference speed.
  arXiv  Detail & Related papers  (2024-08-15T16:45:16Z)
• Non-Federated Multi-Task Split Learning for Heterogeneous Sources [17.47679789733922]
  We introduce a new architecture and methodology to perform multi-task learning for heterogeneous data sources efficiently. We show through theoretical analysis that MTSL can achieve fast convergence by tuning the learning rate of the server and clients.
  arXiv  Detail & Related papers  (2024-05-31T19:27:03Z)
• FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharing [14.938531944702193]
  We propose Federated Learning with Local Heterogeneous Sharing (FedLPS) FedLPS uses transfer learning to facilitate the deployment of multiple tasks on a single device by dividing the local model into a shareable encoder and task-specific encoders. FedLPS significantly outperforms the state-of-the-art (SOTA) FL frameworks by up to 4.88% and reduces the computational resource consumption by 21.3%.
  arXiv  Detail & Related papers  (2024-02-13T16:30:30Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
  We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data. Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
  arXiv  Detail & Related papers  (2023-10-21T12:07:04Z)
• Bridging the Gap Between Foundation Models and Heterogeneous Federated Learning [9.198799314774437]
  Federated learning (FL) offers privacy-preserving decentralized machine learning, optimizing models at edge clients without sharing private data. Foundation models (FMs) have gained traction in the artificial intelligence (AI) community due to their exceptional performance across various tasks. We present an adaptive framework for Resource-aware Federated Foundation Models (RaFFM) to address these challenges.
  arXiv  Detail & Related papers  (2023-09-30T04:31:53Z)
• REFT: Resource-Efficient Federated Training Framework for Heterogeneous and Resource-Constrained Environments [2.117841684082203]
  Federated Learning (FL) plays a critical role in distributed systems. FL emerges as a privacy-enforcing sub-domain of machine learning. We propose "Resource-Efficient Federated Training Framework for Heterogeneous and Resource-Constrained Environments"
  arXiv  Detail & Related papers  (2023-08-25T20:33:30Z)
• Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity [72.85248553787538]
  This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE) Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
  arXiv  Detail & Related papers  (2023-08-04T04:18:01Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Dynamic Attention-based Communication-Efficient Federated Learning [85.18941440826309]
  Federated learning (FL) offers a solution to train a global machine learning model. FL suffers performance degradation when client data distribution is non-IID. We propose a new adaptive training algorithm $textttAdaFL$ to combat this degradation.
  arXiv  Detail & Related papers  (2021-08-12T14:18:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.