Forget-SVGD: Particle-Based Bayesian Federated Unlearning
- URL: http://arxiv.org/abs/2111.12056v1
- Date: Tue, 23 Nov 2021 18:15:50 GMT
- Title: Forget-SVGD: Particle-Based Bayesian Federated Unlearning
- Authors: Jinu Gong, Osvaldo Simeone, Rahif Kassab, and Joonhyuk Kang
- Abstract summary: Forget-Stein Variational Gradient Descent (Forget-SVGD) builds on SVGD.
The proposed method is validated via performance comparisons with non-parametric schemes that train from scratch by excluding data to be forgotten.
- Score: 32.638916321653554
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Variational particle-based Bayesian learning methods have the advantage of
not being limited by the bias affecting more conventional parametric
techniques. This paper proposes to leverage the flexibility of non-parametric
Bayesian approximate inference to develop a novel Bayesian federated unlearning
method, referred to as Forget-Stein Variational Gradient Descent (Forget-SVGD).
Forget-SVGD builds on SVGD - a particle-based approximate Bayesian inference
scheme using gradient-based deterministic updates - and on its distributed
(federated) extension known as Distributed SVGD (DSVGD). Upon the completion of
federated learning, as one or more participating agents request for their data
to be "forgotten", Forget-SVGD carries out local SVGD updates at the agents
whose data need to be "unlearned", which are interleaved with communication
rounds with a parameter server. The proposed method is validated via
performance comparisons with non-parametric schemes that train from scratch by
excluding data to be forgotten, as well as with existing parametric Bayesian
unlearning methods.
Related papers
- IPoD: Implicit Field Learning with Point Diffusion for Generalizable 3D Object Reconstruction from Single RGB-D Images [50.4538089115248]
Generalizable 3D object reconstruction from single-view RGB-D images remains a challenging task.
We propose a novel approach, IPoD, which harmonizes implicit field learning with point diffusion.
Experiments conducted on the CO3D-v2 dataset affirm the superiority of IPoD, achieving 7.8% improvement in F-score and 28.6% in Chamfer distance over existing methods.
arXiv Detail & Related papers (2024-03-30T07:17:37Z) - Bayesian Diffusion Models for 3D Shape Reconstruction [54.69889488052155]
We present a prediction algorithm that performs effective Bayesian inference by tightly coupling the top-down (prior) information with the bottom-up (data-driven) procedure.
We show the effectiveness of BDM on the 3D shape reconstruction task.
arXiv Detail & Related papers (2024-03-11T17:55:53Z) - Accelerating Convergence of Stein Variational Gradient Descent via Deep
Unfolding [5.584060970507506]
Stein variational gradient descent (SVGD) is a prominent particle-based variational inference method used for sampling a target distribution.
In this paper, we propose novel trainable algorithms that incorporate a deep-learning technique called deep unfolding,into SVGD.
arXiv Detail & Related papers (2024-02-23T06:24:57Z) - Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud
Semantic Segmentation via Decoupling Optimization [64.36097398869774]
Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding.
The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data.
We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
arXiv Detail & Related papers (2024-01-13T04:16:40Z) - 3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking [15.330384668966806]
State-of-the-art 3D multi-object tracking (MOT) approaches typically rely on non-learned model-based algorithms such as Kalman Filter.
We propose 3DMOTFormer, a learned geometry-based 3D MOT framework building upon the transformer architecture.
Our approach achieves 71.2% and 68.2% AMOTA on the nuScenes validation and test split, respectively.
arXiv Detail & Related papers (2023-08-12T19:19:58Z) - On Stein Variational Neural Network Ensembles [8.178886940116035]
In this work, we study different Stein variational gradient descent (SVGD) methods operating in the weight space, function space, and in a hybrid setting.
We find that SVGD using functional and hybrid kernels can overcome the limitations of deep ensembles.
arXiv Detail & Related papers (2021-06-20T21:52:46Z) - Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes [86.2129580231191]
Adjoint Rigid Transform (ART) Network is a neural module which can be integrated with a variety of 3D networks.
ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks.
We will release our code and pre-trained models for further research.
arXiv Detail & Related papers (2021-02-01T20:58:45Z) - Facilitate the Parametric Dimension Reduction by Gradient Clipping [1.9671123873378715]
We extend a well-known dimension reduction method, t-distributed neighbor embedding (t-SNE), from non-parametric to parametric by training neural networks.
Our method achieves an embedding quality that is compatible with the non-parametric t-SNE while enjoying the ability of generalization.
arXiv Detail & Related papers (2020-09-30T01:21:22Z) - Federated Generalized Bayesian Learning via Distributed Stein
Variational Gradient Descent [38.41707037232561]
This paper introduces Distributed Stein Variational Gradient Descent (DSVGD), a non-parametric generalized Bayesian inference framework for federated learning.
By varying the number of particles, DSVGD enables a flexible trade-off between per-iteration communication load and number of communication rounds.
arXiv Detail & Related papers (2020-09-11T17:33:22Z) - Kernel Stein Generative Modeling [68.03537693810972]
Gradient Langevin Dynamics (SGLD) demonstrates impressive results with energy-based models on high-dimensional and complex data distributions.
Stein Variational Gradient Descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate a given distribution.
We propose noise conditional kernel SVGD (NCK-SVGD), that works in tandem with the recently introduced Noise Conditional Score Network estimator.
arXiv Detail & Related papers (2020-07-06T21:26:04Z) - Stein Variational Inference for Discrete Distributions [70.19352762933259]
We propose a simple yet general framework that transforms discrete distributions to equivalent piecewise continuous distributions.
Our method outperforms traditional algorithms such as Gibbs sampling and discontinuous Hamiltonian Monte Carlo.
We demonstrate that our method provides a promising tool for learning ensembles of binarized neural network (BNN)
In addition, such transform can be straightforwardly employed in gradient-free kernelized Stein discrepancy to perform goodness-of-fit (GOF) test on discrete distributions.
arXiv Detail & Related papers (2020-03-01T22:45:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.