Federated Coordinate Descent for Privacy-Preserving Multiparty Linear
Regression
- URL: http://arxiv.org/abs/2209.07702v2
- Date: Mon, 19 Sep 2022 08:28:36 GMT
- Title: Federated Coordinate Descent for Privacy-Preserving Multiparty Linear
Regression
- Authors: Xinlin Leng, Chenxu Li, Hongtao Wang
- Abstract summary: We present Federated Coordinate Descent, a new distributed scheme called FCD, to address this issue securely under multiparty scenarios.
Specifically, through secure aggregation and added perturbations, our scheme guarantees that: (1) no local information is leaked to other parties, and (2) global model parameters are not exposed to cloud servers.
We show that the FCD scheme fills the gap of multiparty secure Coordinate Descent methods and is applicable for general linear regressions, including linear, ridge and lasso regressions.
- Score: 0.5049057348282932
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Distributed privacy-preserving regression schemes have been developed and
extended in various fields, where multiparty collaboratively and privately run
optimization algorithms, e.g., Gradient Descent, to learn a set of optimal
parameters. However, traditional Gradient-Descent based methods fail to solve
problems which contains objective functions with L1 regularization, such as
Lasso regression. In this paper, we present Federated Coordinate Descent, a new
distributed scheme called FCD, to address this issue securely under multiparty
scenarios. Specifically, through secure aggregation and added perturbations,
our scheme guarantees that: (1) no local information is leaked to other
parties, and (2) global model parameters are not exposed to cloud servers. The
added perturbations can eventually be eliminated by each party to derive a
global model with high performance. We show that the FCD scheme fills the gap
of multiparty secure Coordinate Descent methods and is applicable for general
linear regressions, including linear, ridge and lasso regressions. Theoretical
security analysis and experimental results demonstrate that FCD can be
performed effectively and efficiently, and provide as low MAE measure as
centralized methods under tasks of three types of linear regressions on
real-world UCI datasets.
Related papers
- Joint Demonstration and Preference Learning Improves Policy Alignment with Human Feedback [58.049113055986375]
We develop a single stage approach named Alignment with Integrated Human Feedback (AIHF) to train reward models and the policy.
The proposed approach admits a suite of efficient algorithms, which can easily reduce to, and leverage, popular alignment algorithms.
We demonstrate the efficiency of the proposed solutions with extensive experiments involving alignment problems in LLMs and robotic control problems in MuJoCo.
arXiv Detail & Related papers (2024-06-11T01:20:53Z) - Adaptive debiased SGD in high-dimensional GLMs with streaming data [4.704144189806667]
We introduce a novel approach to online inference in high-dimensional generalized linear models.
Our method operates in a single-pass mode, significantly reducing both time and space complexity.
We demonstrate that our method, termed the Approximated Debiased Lasso (ADL), not only mitigates the need for the bounded individual probability condition but also significantly improves numerical performance.
arXiv Detail & Related papers (2024-05-28T15:36:48Z) - GIFD: A Generative Gradient Inversion Method with Feature Domain
Optimization [52.55628139825667]
Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy.
Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge.
We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
arXiv Detail & Related papers (2023-08-09T04:34:21Z) - Differentially private sliced inverse regression in the federated
paradigm [3.539008590223188]
We extend Sliced inverse regression (SIR) to address the challenges of decentralized data, prioritizing privacy and communication efficiency.
Our approach, named as federated sliced inverse regression (FSIR), facilitates collaborative estimation of the sufficient dimension reduction subspace among multiple clients.
arXiv Detail & Related papers (2023-06-10T00:32:39Z) - Provable Offline Preference-Based Reinforcement Learning [95.00042541409901]
We investigate the problem of offline Preference-based Reinforcement Learning (PbRL) with human feedback.
We consider the general reward setting where the reward can be defined over the whole trajectory.
We introduce a new single-policy concentrability coefficient, which can be upper bounded by the per-trajectory concentrability.
arXiv Detail & Related papers (2023-05-24T07:11:26Z) - Federated Sufficient Dimension Reduction Through High-Dimensional Sparse
Sliced Inverse Regression [4.561305216067566]
Federated learning has become a popular tool in the big data era nowadays.
We propose a federated sparse sliced inverse regression algorithm for the first time.
arXiv Detail & Related papers (2023-01-23T15:53:06Z) - Offline Policy Optimization in RL with Variance Regularizaton [142.87345258222942]
We propose variance regularization for offline RL algorithms, using stationary distribution corrections.
We show that by using Fenchel duality, we can avoid double sampling issues for computing the gradient of the variance regularizer.
The proposed algorithm for offline variance regularization (OVAR) can be used to augment any existing offline policy optimization algorithms.
arXiv Detail & Related papers (2022-12-29T18:25:01Z) - Communication-Efficient Distributed Quantile Regression with Optimal
Statistical Guarantees [2.064612766965483]
We address the problem of how to achieve optimal inference in distributed quantile regression without stringent scaling conditions.
The difficulties are resolved through a double-smoothing approach that is applied to the local (at each data source) and global objective functions.
Despite the reliance on a delicate combination of local and global smoothing parameters, the quantile regression model is fully parametric.
arXiv Detail & Related papers (2021-10-25T17:09:59Z) - Learning Invariant Representations and Risks for Semi-supervised Domain
Adaptation [109.73983088432364]
We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA)
We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
arXiv Detail & Related papers (2020-10-09T15:42:35Z) - Combining Deep Learning and Optimization for Security-Constrained
Optimal Power Flow [94.24763814458686]
Security-constrained optimal power flow (SCOPF) is fundamental in power systems.
Modeling of APR within the SCOPF problem results in complex large-scale mixed-integer programs.
This paper proposes a novel approach that combines deep learning and robust optimization techniques.
arXiv Detail & Related papers (2020-07-14T12:38:21Z) - Robust Locality-Aware Regression for Labeled Data Classification [5.432221650286726]
We propose a new discriminant feature extraction framework, namely Robust Locality-Aware Regression (RLAR)
In our model, we introduce a retargeted regression to perform the marginal representation learning adaptively instead of using the general average inter-class margin.
To alleviate the disturbance of outliers and prevent overfitting, we measure the regression term and locality-aware term together with the regularization term by the L2,1 norm.
arXiv Detail & Related papers (2020-06-15T11:36:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.