A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Data

• URL: http://arxiv.org/abs/2411.07889v1
• Date: Tue, 12 Nov 2024 15:51:35 GMT
• Title: A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Data
• Authors: Devansh Gupta, A. S. Poornash, Andrew Lowy, Meisam Razaviyayn,
• Abstract summary: We develop a novel algorithm for private and fair federated learning (FL) Our algorithm satisfies inter-silo record-level differential privacy (ISRL-DP) Experiments demonstrate the state-of-the-art fairness-accuracy framework tradeoffs of our algorithm across different privacy levels.
• Score: 14.748203847227542
• License:
• Abstract: Machine learning models are often trained on sensitive data (e.g., medical records and race/gender) that is distributed across different "silos" (e.g., hospitals). These federated learning models may then be used to make consequential decisions, such as allocating healthcare resources. Two key challenges emerge in this setting: (i) maintaining the privacy of each person's data, even if other silos or an adversary with access to the central server tries to infer this data; (ii) ensuring that decisions are fair to different demographic groups (e.g., race/gender). In this paper, we develop a novel algorithm for private and fair federated learning (FL). Our algorithm satisfies inter-silo record-level differential privacy (ISRL-DP), a strong notion of private FL requiring that silo i's sent messages satisfy record-level differential privacy for all i. Our framework can be used to promote different fairness notions, including demographic parity and equalized odds. We prove that our algorithm converges under mild smoothness assumptions on the loss function, whereas prior work required strong convexity for convergence. As a byproduct of our analysis, we obtain the first convergence guarantee for ISRL-DP nonconvex-strongly concave min-max FL. Experiments demonstrate the state-of-the-art fairness-accuracy tradeoffs of our algorithm across different privacy levels.

Related papers

• Targeted Learning for Data Fairness [52.59573714151884]
  We expand fairness inference by evaluating fairness in the data generating process itself. We derive estimators demographic parity, equal opportunity, and conditional mutual information. To validate our approach, we perform several simulations and apply our estimators to real data.
  arXiv  Detail & Related papers  (2025-02-06T18:51:28Z)
• Differentially Private Random Feature Model [52.468511541184895]
  We produce a differentially private random feature model for privacy-preserving kernel machines. We show that our method preserves privacy and derive a generalization error bound for the method.
  arXiv  Detail & Related papers  (2024-12-06T05:31:08Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Randomized Quantization is All You Need for Differential Privacy in Federated Learning [1.9785872350085876]
  We consider an approach to federated learning that combines quantization and differential privacy. We develop a new algorithm called the textbfRandomized textbfQuantization textbfMechanism (RQM) We empirically study the performance of our algorithm and demonstrate that compared to previous work it yields improved privacy-accuracy trade-offs.
  arXiv  Detail & Related papers  (2023-06-20T21:54:13Z)
• Benchmarking FedAvg and FedCurv for Image Classification Tasks [1.376408511310322]
  This paper focuses on the problem of statistical heterogeneity of the data in the same federated network. Several Federated Learning algorithms, such as FedAvg, FedProx and Federated Curvature (FedCurv) have already been proposed. As a side product of this work, we release the non-IID version of the datasets we used so to facilitate further comparisons from the FL community.
  arXiv  Detail & Related papers  (2023-03-31T10:13:01Z)
• Stochastic Differentially Private and Fair Learning [7.971065005161566]
  We provide the first differentially private algorithm for fair learning that is guaranteed to converge. Our framework is flexible enough to permit different fairness, including demographic parity and equalized odds. Our algorithm can be applied to non-binary classification tasks with multiple (non-binary) sensitive attributes.
  arXiv  Detail & Related papers  (2022-10-17T06:54:57Z)
• Differentially Private Vertical Federated Clustering [13.27934054846057]
  In many applications, multiple parties have private data regarding the same set of users but on disjoint sets of attributes. To enable model learning while protecting the privacy of the data subjects, we need vertical federated learning (VFL) techniques. The algorithm proposed in this paper is the first practical solution for differentially private vertical federated k-means clustering.
  arXiv  Detail & Related papers  (2022-08-02T19:23:48Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Differentially Private Federated Learning on Heterogeneous Data [10.431137628048356]
  Federated Learning (FL) is a paradigm for large-scale distributed learning. It faces two key challenges: (i) efficient training from highly heterogeneous user data, and (ii) protecting the privacy of participating users. We propose a novel FL approach to tackle these two challenges together by incorporating Differential Privacy (DP) constraints.
  arXiv  Detail & Related papers  (2021-11-17T18:23:49Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.