Related papers: On the utility and protection of optimization with differential privacy and classic regularization techniques

On the utility and protection of optimization with differential privacy and classic regularization techniques

URL: http://arxiv.org/abs/2209.03175v1
Date: Wed, 7 Sep 2022 14:10:21 GMT
Title: On the utility and protection of optimization with differential privacy and classic regularization techniques
Authors: Eugenio Lomurno, Matteo matteucci
Abstract summary: We study the effectiveness of the differentially-private descent (DP-SGD) algorithm against standard optimization practices with regularization techniques. We discuss differential privacy's flaws and limits and empirically demonstrate the often superior privacy-preserving properties of dropout and l2-regularization.
Score: 9.413131350284083
License: http://creativecommons.org/licenses/by-sa/4.0/
Abstract: Nowadays, owners and developers of deep learning models must consider stringent privacy-preservation rules of their training data, usually crowd-sourced and retaining sensitive information. The most widely adopted method to enforce privacy guarantees of a deep learning model nowadays relies on optimization techniques enforcing differential privacy. According to the literature, this approach has proven to be a successful defence against several models' privacy attacks, but its downside is a substantial degradation of the models' performance. In this work, we compare the effectiveness of the differentially-private stochastic gradient descent (DP-SGD) algorithm against standard optimization practices with regularization techniques. We analyze the resulting models' utility, training performance, and the effectiveness of membership inference and model inversion attacks against the learned models. Finally, we discuss differential privacy's flaws and limits and empirically demonstrate the often superior privacy-preserving properties of dropout and l2-regularization.