Federated Learning with Convex Global and Local Constraints

• URL: http://arxiv.org/abs/2310.10117v3
• Date: Wed, 1 May 2024 10:11:15 GMT
• Title: Federated Learning with Convex Global and Local Constraints
• Authors: Chuan He, Le Peng, Ju Sun,
• Abstract summary: In practice, many machine learning (ML) problems come with constraints, their applied domains involve distributed sensitive data that cannot be shared with others. We propose a new FL algorithm for ML problems based on the proximal Lagrangian (AL) method.
• Score: 4.094311966028137
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In practice, many machine learning (ML) problems come with constraints, and their applied domains involve distributed sensitive data that cannot be shared with others, e.g., in healthcare. Collaborative learning in such practical scenarios entails federated learning (FL) for ML problems with constraints, or FL with constraints for short. Despite the extensive developments of FL techniques in recent years, these techniques only deal with unconstrained FL problems or FL problems with simple constraints that are amenable to easy projections. There is little work dealing with FL problems with general constraints. To fill this gap, we take the first step toward building an algorithmic framework for solving FL problems with general constraints. In particular, we propose a new FL algorithm for constrained ML problems based on the proximal augmented Lagrangian (AL) method. Assuming convex objective and convex constraints plus other mild conditions, we establish the worst-case complexity of the proposed algorithm. Our numerical experiments show the effectiveness of our algorithm in performing Neyman-Pearson classification and fairness-aware learning with nonconvex constraints, in an FL setting.

Related papers

• Single-loop Algorithms for Stochastic Non-convex Optimization with Weakly-Convex Constraints [49.76332265680669]
  This paper examines a crucial subset of problems where both the objective and constraint functions are weakly convex. Existing methods often face limitations, including slow convergence rates or reliance on double-loop designs. We introduce a novel single-loop penalty-based algorithm to overcome these challenges.
  arXiv  Detail & Related papers  (2025-04-21T17:15:48Z)
• Where is the Testbed for my Federated Learning Research? [3.910931245706272]
  We present CoLExT, a real-world testbed for federated learning (FL) research. CoLExT is designed to streamline experimentation with custom FL algorithms in a rich testbed configuration space. Through an initial investigation involving popular FL algorithms running on CoLExT, we reveal previously unknown trade-offs, inefficiencies, and programming bugs.
  arXiv  Detail & Related papers  (2024-07-19T09:34:04Z)
• Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm [8.479947546216131]
  We develop a Hessian-free gradient-based algorithm-termed proximal Lagrangian Value function-based Hessian-free Bi-level Algorithm (LV-HBA) LV-HBA is especially well-suited for machine learning applications.
  arXiv  Detail & Related papers  (2024-01-29T13:50:56Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Automated Federated Learning in Mobile Edge Networks -- Fast Adaptation and Convergence [83.58839320635956]
  Federated Learning (FL) can be used in mobile edge networks to train machine learning models in a distributed manner. Recent FL has been interpreted within a Model-Agnostic Meta-Learning (MAML) framework, which brings FL significant advantages in fast adaptation and convergence over heterogeneous datasets. This paper addresses how much benefit MAML brings to FL and how to maximize such benefit over mobile edge networks.
  arXiv  Detail & Related papers  (2023-03-23T02:42:10Z)
• FLAGS Framework for Comparative Analysis of Federated Learning Algorithms [0.0]
  This work consolidates the Federated Learning landscape and offers an objective analysis of the major FL algorithms. To enable a uniform assessment, a multi-FL framework named FLAGS: Federated Learning AlGorithms Simulation has been developed. Our experiments indicate that fully decentralized FL algorithms achieve comparable accuracy under multiple operating conditions.
  arXiv  Detail & Related papers  (2022-12-14T12:08:30Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning [4.492444446637857]
  Federated learning is an increasingly popular machine learning paradigm in which multiple nodes try to collaboratively learn. Standard average risk minimization of supervised learning is inadequate in handling several major constraints specific to federated learning. We introduce a new framework, FLIX, that takes into account the unique challenges brought by federated learning.
  arXiv  Detail & Related papers  (2021-11-22T22:06:58Z)
• Compositional federated learning: Applications in distributionally robust averaging and meta learning [31.97853929292195]
  We propose an effective and efficient Compositional Federated Learning (ComFedL) algorithm for solving a new compositional Federated Learning (FL) framework. We prove that our ComFedL algorithm achieves a convergence rate of $O(frac1sqrtT)$, where $T$ denotes the number of iteration.
  arXiv  Detail & Related papers  (2021-06-21T17:08:09Z)
• Decentralized Personalized Federated Learning for Min-Max Problems [79.61785798152529]
  This paper is the first to study PFL for saddle point problems encompassing a broader range of optimization problems. We propose new algorithms to address this problem and provide a theoretical analysis of the smooth (strongly) convex-(strongly) concave saddle point problems. Numerical experiments for bilinear problems and neural networks with adversarial noise demonstrate the effectiveness of the proposed methods.
  arXiv  Detail & Related papers  (2021-06-14T10:36:25Z)
• Constrained Learning with Non-Convex Losses [119.8736858597118]
  Though learning has become a core technology of modern information processing, there is now ample evidence that it can lead to biased, unsafe, and prejudiced solutions.
  arXiv  Detail & Related papers  (2021-03-08T23:10:33Z)
• Delay Minimization for Federated Learning Over Wireless Communication Networks [172.42768672943365]
  The problem of delay computation for federated learning (FL) over wireless communication networks is investigated. A bisection search algorithm is proposed to obtain the optimal solution. Simulation results show that the proposed algorithm can reduce delay by up to 27.3% compared to conventional FL methods.
  arXiv  Detail & Related papers  (2020-07-05T19:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.