Multiple Kernel-Based Online Federated Learning

• URL: http://arxiv.org/abs/2102.10861v1
• Date: Mon, 22 Feb 2021 09:51:00 GMT
• Title: Multiple Kernel-Based Online Federated Learning
• Abstract summary: We propose a novel multiple kernel-based OFL (MK-OFL) as a non-trivial extension of OMKL. We theoretically prove that MK-OFL achieves the optimal sublinear regret bound when compared with the best function in hindsight.
• Score: 15.102346715690755
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online federated learning (OFL) becomes an emerging learning framework, in which edge nodes perform online learning with continuous streaming local data and a server constructs a global model from the aggregated local models. Online multiple kernel learning (OMKL), using a preselected set of P kernels, can be a good candidate for OFL framework as it has provided an outstanding performance with a low-complexity and scalability. Yet, an naive extension of OMKL into OFL framework suffers from a heavy communication overhead that grows linearly with P. In this paper, we propose a novel multiple kernel-based OFL (MK-OFL) as a non-trivial extension of OMKL, which yields the same performance of the naive extension with 1/P communication overhead reduction. We theoretically prove that MK-OFL achieves the optimal sublinear regret bound when compared with the best function in hindsight. Finally, we provide the numerical tests of our approach on real-world datasets, which suggests its practicality.

Related papers

• Federated Nested Learning: Collaborative Training of Self-Referential Memories for Test-Time Adaptation [50.940432795341735]
  Federated Nested Learning (FedNL) is a novel framework that reformulates Federated Learning as a three-level nested optimization system.<n>FedNL embeds Titans-based linear attention into FL, enabling clients to perform lightweight, zero-shot test-time adaptation.
  arXiv  Detail & Related papers  (2026-05-08T04:31:46Z)
• LoLaFL: Low-Latency Federated Learning via Forward-only Propagation [25.99531618965931]
  Federated learning (FL) has emerged as a widely adopted paradigm for enabling edge learning with distributed data. Traditional FL with deep neural networks trained via backpropagation can hardly meet the low-latency learning requirements in the sixth generation (6G) mobile networks. We adopt the state-of-the-art principle of maximal coding rate reduction to learn linear discriminative features and extend the resultant white-box neural network into FL.
  arXiv  Detail & Related papers  (2024-12-19T09:20:27Z)
• DapperFL: Domain Adaptive Federated Learning with Model Fusion Pruning for Edge Devices [43.06053432074504]
  Federated learning (FL) has emerged as a prominent machine learning paradigm in edge computing environments. We propose a heterogeneous FL framework DapperFL to enhance model performance across multiple domains. We implement DapperFL on a realworld FL platform with heterogeneous clients.
  arXiv  Detail & Related papers  (2024-12-08T05:50:04Z)
• SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead [75.87007729801304]
  SpaFL: a communication-efficient FL framework is proposed to optimize sparse model structures with low computational overhead. Experiments show that SpaFL improves accuracy while requiring much less communication and computing resources compared to sparse baselines.
  arXiv  Detail & Related papers  (2024-06-01T13:10:35Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Delay-Aware Hierarchical Federated Learning [7.292078085289465]
  The paper introduces delay-aware hierarchical federated learning (DFL) to improve the efficiency of distributed machine learning (ML) model training. During global synchronization, the cloud server consolidates local models with an outdated global model using a convex control algorithm. Numerical evaluations show DFL's superior performance in terms of faster global model, reduced convergence resource, and evaluations against communication delays.
  arXiv  Detail & Related papers  (2023-03-22T09:23:29Z)
• Hierarchical Personalized Federated Learning Over Massive Mobile Edge Computing Networks [95.39148209543175]
  We propose hierarchical PFL (HPFL), an algorithm for deploying PFL over massive MEC networks. HPFL combines the objectives of training loss minimization and round latency minimization while jointly determining the optimal bandwidth allocation.
  arXiv  Detail & Related papers  (2023-03-19T06:00:05Z)
• Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$-Means Clustering [24.51727660771511]
  We propose a unified framework by constructing coresets in a distributed fashion for communication-efficient VFL. We study two important learning tasks in the VFL setting: regularized linear regression and $k$-means clustering. We theoretically show that using coresets can drastically alleviate the communication complexity, while nearly maintain the solution quality.
  arXiv  Detail & Related papers  (2022-10-26T12:28:53Z)
• Vertical Semi-Federated Learning for Efficient Online Advertising [50.18284051956359]
  Semi-VFL (Vertical Semi-Federated Learning) is proposed to achieve a practical industry application fashion for VFL. We build an inference-efficient single-party student model applicable to the whole sample space. New representation distillation methods are designed to extract cross-party feature correlations for both the overlapped and non-overlapped data.
  arXiv  Detail & Related papers  (2022-09-30T17:59:27Z)
• On-the-fly Resource-Aware Model Aggregation for Federated Learning in Heterogeneous Edge [15.932747809197517]
  Edge computing has revolutionized the world of mobile and wireless networks world thanks to its flexible, secure, and performing characteristics. In this paper, we conduct an in-depth study of strategies to replace a central aggregation server with a flying master. Our results demonstrate a significant reduction of runtime using our flying master FL framework compared to the original FL from measurements results conducted in our EdgeAI testbed and over real 5G networks.
  arXiv  Detail & Related papers  (2021-12-21T19:04:42Z)
• Context-Aware Online Client Selection for Hierarchical Federated Learning [33.205640790962505]
  Federated Learning (FL) has been considered as an appealing framework to tackle data privacy issues. Federated Learning (FL) has been considered as an appealing framework to tackle data privacy issues.
  arXiv  Detail & Related papers  (2021-12-02T01:47:01Z)
• Wireless Communications for Collaborative Federated Learning [160.82696473996566]
  Internet of Things (IoT) devices may not be able to transmit their collected data to a central controller for training machine learning models. Google's seminal FL algorithm requires all devices to be directly connected with a central controller. This paper introduces a novel FL framework, called collaborative FL (CFL), which enables edge devices to implement FL with less reliance on a central controller.
  arXiv  Detail & Related papers  (2020-06-03T20:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.