Advice Conformance Verification by Reinforcement Learning agents for Human-in-the-Loop

• URL: http://arxiv.org/abs/2210.03455v1
• Date: Fri, 7 Oct 2022 10:56:28 GMT
• Title: Advice Conformance Verification by Reinforcement Learning agents for Human-in-the-Loop
• Authors: Mudit Verma, Ayush Kharkwal, Subbarao Kambhampati
• Abstract summary: We study two cases of good and bad advice scenarios in MuJoCo's Humanoid environment. We show that our method can provide an interpretable means of solving the Advice-Conformance Verification problem.
• Score: 17.042179951736262
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human-in-the-loop (HiL) reinforcement learning is gaining traction in domains with large action and state spaces, and sparse rewards by allowing the agent to take advice from HiL. Beyond advice accommodation, a sequential decision-making agent must be able to express the extent to which it was able to utilize the human advice. Subsequently, the agent should provide a means for the HiL to inspect parts of advice that it had to reject in favor of the overall environment objective. We introduce the problem of Advice-Conformance Verification which requires reinforcement learning (RL) agents to provide assurances to the human in the loop regarding how much of their advice is being conformed to. We then propose a Tree-based lingua-franca to support this communication, called a Preference Tree. We study two cases of good and bad advice scenarios in MuJoCo's Humanoid environment. Through our experiments, we show that our method can provide an interpretable means of solving the Advice-Conformance Verification problem by conveying whether or not the agent is using the human's advice. Finally, we present a human-user study with 20 participants that validates our method.

Related papers

• Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
  In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent. We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents. Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
  arXiv  Detail & Related papers  (2024-06-16T15:24:50Z)
• ADESSE: Advice Explanations in Complex Repeated Decision-Making Environments [14.105935964906976]
  This work considers a problem setup where an intelligent agent provides advice to a human decision-maker. We develop an approach named ADESSE to generate explanations about the adviser agent to improve human trust and decision-making.
  arXiv  Detail & Related papers  (2024-05-31T08:59:20Z)
• Robustifying a Policy in Multi-Agent RL with Diverse Cooperative Behaviors and Adversarial Style Sampling for Assistive Tasks [51.00472376469131]
  We propose a framework that learns a robust caregiver's policy by training it for diverse care-receiver responses. We demonstrate that policies trained with a popular deep RL method are vulnerable to changes in policies of other agents.
  arXiv  Detail & Related papers  (2024-03-01T08:15:18Z)
• RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agents [30.250555783628762]
  This research argues that addressing these issues is not solely the recommender systems' responsibility. We introduce the RAH Recommender system, Assistant, and Human framework, emphasizing the alignment with user personalities. Our contributions provide a human-centered recommendation framework that partners effectively with various recommendation models.
  arXiv  Detail & Related papers  (2023-08-19T04:46:01Z)
• Learning When to Advise Human Decision Makers [12.47847261193524]
  We propose a novel design of AI systems in which the algorithm interacts with the human user in a two-sided manner. The results of a large-scale experiment show that our advising approach manages to provide advice at times of need.
  arXiv  Detail & Related papers  (2022-09-27T17:52:13Z)
• Teachable Reinforcement Learning via Advice Distillation [161.43457947665073]
  We propose a new supervision paradigm for interactive learning based on "teachable" decision-making systems that learn from structured advice provided by an external teacher. We show that agents that learn from advice can acquire new skills with significantly less human supervision than standard reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-03-19T03:22:57Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)
• Robust Reinforcement Learning on State Observations with Learned Optimal Adversary [86.0846119254031]
  We study the robustness of reinforcement learning with adversarially perturbed state observations. With a fixed agent policy, we demonstrate that an optimal adversary to perturb state observations can be found. For DRL settings, this leads to a novel empirical adversarial attack to RL agents via a learned adversary that is much stronger than previous ones.
  arXiv  Detail & Related papers  (2021-01-21T05:38:52Z)
• Human Engagement Providing Evaluative and Informative Advice for Interactive Reinforcement Learning [2.5799044614524664]
  This work focuses on answering which of two approaches, evaluative or informative, is the preferred instructional approach for humans. Results show users giving informative advice provide more accurate advice, are willing to assist the learner agent for a longer time, and provide more advice per episode.
  arXiv  Detail & Related papers  (2020-09-21T02:14:02Z)
• Self-Supervised Reinforcement Learning for Recommender Systems [77.38665506495553]
  We propose self-supervised reinforcement learning for sequential recommendation tasks. Our approach augments standard recommendation models with two output layers: one for self-supervised learning and the other for RL. Based on such an approach, we propose two frameworks namely Self-Supervised Q-learning(SQN) and Self-Supervised Actor-Critic(SAC)
  arXiv  Detail & Related papers  (2020-06-10T11:18:57Z)
• A two-level solution to fight against dishonest opinions in recommendation-based trust systems [13.356755375091456]
  We consider a scenario in which an agent requests recommendations from multiple parties to build trust toward another agent. At the collection level, we propose to allow agents to self-assess the accuracy of their recommendations. At the processing level, we propose a recommendations aggregation technique that is resilient to collusion attacks.
  arXiv  Detail & Related papers  (2020-06-09T00:34:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.