PAPER-HILT: Personalized and Adaptive Privacy-Aware Early-Exit for Reinforcement Learning in Human-in-the-Loop Systems

• URL: http://arxiv.org/abs/2403.05864v1
• Date: Sat, 9 Mar 2024 10:24:12 GMT
• Title: PAPER-HILT: Personalized and Adaptive Privacy-Aware Early-Exit for Reinforcement Learning in Human-in-the-Loop Systems
• Authors: Mojtaba Taherisadr, Salma Elmalaki
• Abstract summary: Reinforcement Learning (RL) has increasingly become a preferred method over traditional rule-based systems in diverse human-in-the-loop (HITL) applications. This paper focuses on developing an innovative, adaptive RL strategy through exploiting an early-exit approach designed explicitly for privacy preservation in HITL environments.
• Score: 0.6282068591820944
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement Learning (RL) has increasingly become a preferred method over traditional rule-based systems in diverse human-in-the-loop (HITL) applications due to its adaptability to the dynamic nature of human interactions. However, integrating RL in such settings raises significant privacy concerns, as it might inadvertently expose sensitive user information. Addressing this, our paper focuses on developing PAPER-HILT, an innovative, adaptive RL strategy through exploiting an early-exit approach designed explicitly for privacy preservation in HITL environments. This approach dynamically adjusts the tradeoff between privacy protection and system utility, tailoring its operation to individual behavioral patterns and preferences. We mainly highlight the challenge of dealing with the variable and evolving nature of human behavior, which renders static privacy models ineffective. PAPER-HILT's effectiveness is evaluated through its application in two distinct contexts: Smart Home environments and Virtual Reality (VR) Smart Classrooms. The empirical results demonstrate PAPER-HILT's capability to provide a personalized equilibrium between user privacy and application utility, adapting effectively to individual user needs and preferences. On average for both experiments, utility (performance) drops by 24%, and privacy (state prediction) improves by 31%.

Related papers

• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• Differentially Private Model-Based Offline Reinforcement Learning [51.1231068185106]
  We introduce DP-MORL, an algorithm coming with differential privacy guarantees. A private model of the environment is first learned from offline data. We then use model-based policy optimization to derive a policy from the private model.
  arXiv  Detail & Related papers  (2024-02-08T10:05:11Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• adaPARL: Adaptive Privacy-Aware Reinforcement Learning for Sequential-Decision Making Human-in-the-Loop Systems [0.5414308305392761]
  Reinforcement learning (RL) presents numerous benefits compared to rule-based approaches in various applications. We propose adaPARL, an adaptive approach for privacy-aware RL, especially for human-in-the-loop IoT systems. AdaPARL provides a personalized privacy-utility trade-off depending on human behavior and preference.
  arXiv  Detail & Related papers  (2023-03-07T21:55:22Z)
• Reinforcement Learning from Diverse Human Preferences [68.4294547285359]
  This paper develops a method for crowd-sourcing preference labels and learning from diverse human preferences. The proposed method is tested on a variety of tasks in DMcontrol and Meta-world. It has shown consistent and significant improvements over existing preference-based RL algorithms when learning from diverse feedback.
  arXiv  Detail & Related papers  (2023-01-27T15:18:54Z)
• On the utility and protection of optimization with differential privacy and classic regularization techniques [9.413131350284083]
  We study the effectiveness of the differentially-private descent (DP-SGD) algorithm against standard optimization practices with regularization techniques. We discuss differential privacy's flaws and limits and empirically demonstrate the often superior privacy-preserving properties of dropout and l2-regularization.
  arXiv  Detail & Related papers  (2022-09-07T14:10:21Z)
• Privacy-Preserving Reinforcement Learning Beyond Expectation [6.495883501989546]
  Cyber and cyber-physical systems equipped with machine learning algorithms such as autonomous cars share environments with humans. It is important to align system (or agent) behaviors with the preferences of one or more human users. We consider the case when an agent has to learn behaviors in an unknown environment.
  arXiv  Detail & Related papers  (2022-03-18T21:28:29Z)
• Reinforcement Learning Beyond Expectation [11.428014000851535]
  Cumulative prospect theory (CPT) is a paradigm that has been empirically shown to model a tendency of humans to view gains and losses differently. In this paper, we consider a setting where an autonomous agent has to learn behaviors in an unknown environment. In order to endow the agent with the ability to closely mimic the behavior of human users, we optimize a CPT-based cost.
  arXiv  Detail & Related papers  (2021-03-29T20:35:25Z)
• Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients [54.98496284653234]
  We consider the task of training a policy that maximizes reward while minimizing disclosure of certain sensitive state variables through the actions. We solve this problem by introducing a regularizer based on the mutual information between the sensitive state and the actions. We develop a model-based estimator for optimization of privacy-constrained policies.
  arXiv  Detail & Related papers  (2020-12-30T03:22:35Z)
• Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach [54.32266555843765]
  This paper studies a model that protects the privacy of the individuals sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differential privacy and the use of Lagrangian duality to design neural networks that can accommodate fairness constraints.
  arXiv  Detail & Related papers  (2020-09-26T10:50:33Z)
• Tempered Sigmoid Activations for Deep Learning with Differential Privacy [33.574715000662316]
  We show that the choice of activation function is central to bounding the sensitivity of privacy-preserving deep learning. We achieve new state-of-the-art accuracy on MNIST, FashionMNIST, and CIFAR10 without any modification of the learning procedure fundamentals.
  arXiv  Detail & Related papers  (2020-07-28T13:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.