Related papers: Iterative Sketching for Secure Coded Regression

Iterative Sketching for Secure Coded Regression

URL: http://arxiv.org/abs/2308.04185v2
Date: Sun, 31 Mar 2024 05:11:35 GMT
Title: Iterative Sketching for Secure Coded Regression
Authors: Neophytos Charalambides, Hessam Mahdavifar, Mert Pilanci, Alfred O. Hero III,
Abstract summary: We propose methods for speeding up distributed linear regression. Specifically, we randomly rotate the basis of the system of equations and then subsample blocks, to simultaneously secure the information and reduce the dimension of the regression problem.
Score: 66.53950020718021
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Linear regression is a fundamental and primitive problem in supervised machine learning, with applications ranging from epidemiology to finance. In this work, we propose methods for speeding up distributed linear regression. We do so by leveraging randomized techniques, while also ensuring security and straggler resiliency in asynchronous distributed computing systems. Specifically, we randomly rotate the basis of the system of equations and then subsample blocks, to simultaneously secure the information and reduce the dimension of the regression problem. In our setup, the basis rotation corresponds to an encoded encryption in an approximate gradient coding scheme, and the subsampling corresponds to the responses of the non-straggling servers in the centralized coded computing framework. This results in a distributive iterative stochastic approach for matrix compression and steepest descent.

Related papers

Gradient Coding with Iterative Block Leverage Score Sampling [42.21200677508463]
We generalize the leverage score sampling sketch for $ell$-subspace embeddings, to accommodate sampling subsets of the transformed data. This is then used to derive an approximate coded computing approach for first-order methods.
arXiv Detail & Related papers (2023-08-06T12:22:12Z)
A Linearly Convergent GAN Inversion-based Algorithm for Reverse Engineering of Deceptions [1.2891210250935146]
We propose a novel framework for reverse engineering of deceptions that supposes that the clean data lies in the range of a GAN. For the first time in the literature, we provide deterministic linear convergence guarantees for this problem.
arXiv Detail & Related papers (2023-06-07T20:08:27Z)
Dealing with Collinearity in Large-Scale Linear System Identification Using Gaussian Regression [3.04585143845864]
We consider estimation of networks consisting of several interconnected dynamic systems. We develop a strategy cast in a Bayesian regularization framework where any impulse response is seen as realization of a zero-mean Gaussian process. We design a novel Markov chain Monte Carlo scheme able to reconstruct the impulse responses posterior by efficiently dealing with collinearity.
arXiv Detail & Related papers (2023-02-21T19:35:47Z)
Federated Sufficient Dimension Reduction Through High-Dimensional Sparse Sliced Inverse Regression [4.561305216067566]
Federated learning has become a popular tool in the big data era nowadays. We propose a federated sparse sliced inverse regression algorithm for the first time.
arXiv Detail & Related papers (2023-01-23T15:53:06Z)
Covariance-Free Sparse Bayesian Learning [62.24008859844098]
We introduce a new SBL inference algorithm that avoids explicit inversions of the covariance matrix. Our method can be up to thousands of times faster than existing baselines. We showcase how our new algorithm enables SBL to tractably tackle high-dimensional signal recovery problems.
arXiv Detail & Related papers (2021-05-21T16:20:07Z)
Solving weakly supervised regression problem using low-rank manifold regularization [77.34726150561087]
We solve a weakly supervised regression problem. Under "weakly" we understand that for some training points the labels are known, for some unknown, and for others uncertain due to the presence of random noise or other reasons such as lack of resources. In the numerical section, we applied the suggested method to artificial and real datasets using Monte-Carlo modeling.
arXiv Detail & Related papers (2021-04-13T23:21:01Z)
A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
arXiv Detail & Related papers (2020-03-18T05:56:27Z)
Distributed Sketching Methods for Privacy Preserving Regression [54.51566432934556]
We leverage randomized sketches for reducing the problem dimensions as well as preserving privacy and improving straggler resilience in asynchronous distributed systems. We derive novel approximation guarantees for classical sketching methods and analyze the accuracy of parameter averaging for distributed sketches. We illustrate the performance of distributed sketches in a serverless computing platform with large scale experiments.
arXiv Detail & Related papers (2020-02-16T08:35:48Z)
Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving [106.63673243937492]
Feedforward computation, such as evaluating a neural network or sampling from an autoregressive model, is ubiquitous in machine learning. We frame the task of feedforward computation as solving a system of nonlinear equations. We then propose to find the solution using a Jacobi or Gauss-Seidel fixed-point method, as well as hybrid methods of both. Our method is guaranteed to give exactly the same values as the original feedforward computation with a reduced (or equal) number of parallelizable iterations, and hence reduced time given sufficient parallel computing power.
arXiv Detail & Related papers (2020-02-10T10:11:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.