Private Reinforcement Learning with PAC and Regret Guarantees

• URL: http://arxiv.org/abs/2009.09052v1
• Date: Fri, 18 Sep 2020 20:18:35 GMT
• Title: Private Reinforcement Learning with PAC and Regret Guarantees
• Authors: Giuseppe Vietri, Borja Balle, Akshay Krishnamurthy, Zhiwei Steven Wu
• Abstract summary: We design privacy preserving exploration policies for episodic reinforcement learning (RL) We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP) We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee.
• Score: 69.4202374491817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motivated by high-stakes decision-making domains like personalized medicine where user information is inherently sensitive, we design privacy preserving exploration policies for episodic reinforcement learning (RL). We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP)--a strong variant of differential privacy for settings where each user receives their own sets of output (e.g., policy recommendations). We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee. Our algorithm only pays for a moderate privacy cost on exploration: in comparison to the non-private bounds, the privacy parameter only appears in lower-order terms. Finally, we present lower bounds on sample complexity and regret for reinforcement learning subject to JDP.

Related papers

• Enhancing Feature-Specific Data Protection via Bayesian Coordinate Differential Privacy [55.357715095623554]
  Local Differential Privacy (LDP) offers strong privacy guarantees without requiring users to trust external parties. We propose a Bayesian framework, Bayesian Coordinate Differential Privacy (BCDP), that enables feature-specific privacy quantification.
  arXiv  Detail & Related papers  (2024-10-24T03:39:55Z)
• Masked Differential Privacy [64.32494202656801]
  We propose an effective approach called masked differential privacy (DP), which allows for controlling sensitive regions where differential privacy is applied. Our method operates selectively on data and allows for defining non-sensitive-temporal regions without DP application or combining differential privacy with other privacy techniques within data samples.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• A Randomized Approach for Tight Privacy Accounting [63.67296945525791]
  We propose a new differential privacy paradigm called estimate-verify-release (EVR) EVR paradigm first estimates the privacy parameter of a mechanism, then verifies whether it meets this guarantee, and finally releases the query output. Our empirical evaluation shows the newly proposed EVR paradigm improves the utility-privacy tradeoff for privacy-preserving machine learning.
  arXiv  Detail & Related papers  (2023-04-17T00:38:01Z)
• How Do Input Attributes Impact the Privacy Loss in Differential Privacy? [55.492422758737575]
  We study the connection between the per-subject norm in DP neural networks and individual privacy loss. We introduce a novel metric termed the Privacy Loss-Input Susceptibility (PLIS) which allows one to apportion the subject's privacy loss to their input attributes.
  arXiv  Detail & Related papers  (2022-11-18T11:39:03Z)
• Algorithms with More Granular Differential Privacy Guarantees [65.3684804101664]
  We consider partial differential privacy (DP), which allows quantifying the privacy guarantee on a per-attribute basis. In this work, we study several basic data analysis and learning tasks, and design algorithms whose per-attribute privacy parameter is smaller that the best possible privacy parameter for the entire record of a person.
  arXiv  Detail & Related papers  (2022-09-08T22:43:50Z)
• Differentially Private Reinforcement Learning with Linear Function Approximation [3.42658286826597]
  We study regret minimization in finite-horizon Markov decision processes (MDPs) under the constraints of differential privacy (DP) Our results are achieved via a general procedure for learning in linear mixture MDPs under changing regularizers.
  arXiv  Detail & Related papers  (2022-01-18T15:25:24Z)
• Differentially Private Regret Minimization in Episodic Markov Decision Processes [6.396288020763144]
  We study regret in finite horizon tabular Markov decision processes (MDPs) under the constraints of differential privacy (DP) This is motivated by the widespread applications of reinforcement learning (RL) in real-world sequential decision making problems.
  arXiv  Detail & Related papers  (2021-12-20T15:12:23Z)
• Privately Publishable Per-instance Privacy [21.775752827149383]
  We consider how to privately share the personalized privacy losses incurred by objective perturbation, using per-instance differential privacy (pDP) We analyze the per-instance privacy loss of releasing a private empirical risk minimizer learned via objective perturbation, and propose a group of methods to privately and accurately publish the pDP losses at little to no additional privacy cost.
  arXiv  Detail & Related papers  (2021-11-03T15:17:29Z)
• Local Differential Privacy for Regret Minimization in Reinforcement Learning [33.679678503441565]
  We study privacy in the context of finite-horizon Markov Decision Processes (MDPs) We formulate this notion of privacy for RL by leveraging the local differential privacy (LDP) framework. We present an optimistic algorithm that simultaneously satisfies $varepsilon$-LDP requirements.
  arXiv  Detail & Related papers  (2020-10-15T14:13:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.