Towards Interactive Reinforcement Learning with Intrinsic Feedback

• URL: http://arxiv.org/abs/2112.01575v3
• Date: Wed, 23 Aug 2023 17:23:59 GMT
• Title: Towards Interactive Reinforcement Learning with Intrinsic Feedback
• Authors: Benjamin Poole and Minwoo Lee
• Abstract summary: Reinforcement learning (RL) and brain-computer interfaces (BCI) have experienced significant growth over the past decade. With rising interest in human-in-the-loop (HITL), incorporating human input with RL algorithms has given rise to the sub-field of interactive RL. We denote this new and emerging medium of feedback as intrinsic feedback.
• Score: 1.7117805951258132
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) and brain-computer interfaces (BCI) have experienced significant growth over the past decade. With rising interest in human-in-the-loop (HITL), incorporating human input with RL algorithms has given rise to the sub-field of interactive RL. Adjacently, the field of BCI has long been interested in extracting informative brain signals from neural activity for use in human-computer interactions. A key link between these fields lies in the interpretation of neural activity as feedback such that interactive RL approaches can be employed. We denote this new and emerging medium of feedback as intrinsic feedback. Despite intrinsic feedback's ability to be conveyed automatically and even unconsciously, proper exploration surrounding this key link has largely gone unaddressed by both communities. Thus, to help facilitate a deeper understanding and a more effective utilization, we provide a tutorial-style review covering the motivations, approaches, and open problems of intrinsic feedback and its foundational concepts.

Related papers

• A Survey of Reinforcement Learning from Human Feedback [28.92654784501927]
  Reinforcement learning from human feedback (RLHF) is a variant of reinforcement learning (RL) that learns from human feedback instead of relying on an engineered reward function. This article provides a comprehensive overview of the fundamentals of RLHF, exploring the intricate dynamics between RL agents and human input.
  arXiv  Detail & Related papers  (2023-12-22T18:58:06Z)
• Neural-Logic Human-Object Interaction Detection [67.4993347702353]
  We present L OGIC HOI, a new HOI detector that leverages neural-logic reasoning and Transformer to infer feasible interactions between entities. Specifically, we modify the self-attention mechanism in vanilla Transformer, enabling it to reason over the human, action, object> triplet and constitute novel interactions. We formulate these two properties in first-order logic and ground them into continuous space to constrain the learning process of our approach, leading to improved performance and zero-shot generalization capabilities.
  arXiv  Detail & Related papers  (2023-11-16T11:47:53Z)
• Accelerating the Convergence of Human-in-the-Loop Reinforcement Learning with Counterfactual Explanations [1.8275108630751844]
  Human-in-the-loop Reinforcement Learning (HRL) addresses this issue by combining human feedback and reinforcement learning techniques. We extend the existing TAMER Framework with the possibility to enhance human feedback with two different types of counterfactual explanations.
  arXiv  Detail & Related papers  (2021-08-03T08:27:28Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)
• Using Cognitive Models to Train Warm Start Reinforcement Learning Agents for Human-Computer Interactions [6.623676799228969]
  We propose a novel approach of using cognitive models to pre-train RL agents before they are applied to real users. We present our general methodological approach, followed by two case studies from our previous and ongoing projects.
  arXiv  Detail & Related papers  (2021-03-10T16:20:02Z)
• Deep Reinforcement Learning with Interactive Feedback in a Human-Robot Environment [1.2998475032187096]
  We propose a deep reinforcement learning approach with interactive feedback to learn a domestic task in a human-robot scenario. We compare three different learning methods using a simulated robotic arm for the task of organizing different objects. The obtained results show that a learner agent, using either agent-IDeepRL or human-IDeepRL, completes the given task earlier and has fewer mistakes compared to the autonomous DeepRL approach.
  arXiv  Detail & Related papers  (2020-07-07T11:55:27Z)
• Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation [20.837228359591663]
  We present the first study of using human visual explanations in human-in-the-loop reinforcement learning. We propose EXPAND to encourage the model to encode task-relevant features through a context-aware data augmentation.
  arXiv  Detail & Related papers  (2020-06-26T05:40:05Z)
• Cost-effective Interactive Attention Learning with Neural Attention Processes [79.8115563067513]
  We propose a novel interactive learning framework which we refer to as Interactive Attention Learning (IAL) IAL is prone to overfitting due to scarcity of human annotations, and requires costly retraining. We tackle these challenges by proposing a sample-efficient attention mechanism and a cost-effective reranking algorithm for instances and features.
  arXiv  Detail & Related papers  (2020-06-09T17:36:41Z)
• Knowledge-guided Deep Reinforcement Learning for Interactive Recommendation [49.32287384774351]
  Interactive recommendation aims to learn from dynamic interactions between items and users to achieve responsiveness and accuracy. We propose Knowledge-Guided deep Reinforcement learning to harness the advantages of both reinforcement learning and knowledge graphs for interactive recommendation.
  arXiv  Detail & Related papers  (2020-04-17T05:26:47Z)
• Facial Feedback for Reinforcement Learning: A Case Study and Offline Analysis Using the TAMER Framework [51.237191651923666]
  We investigate the potential of agent learning from trainers' facial expressions via interpreting them as evaluative feedback. With designed CNN-RNN model, our analysis shows that telling trainers to use facial expressions and competition can improve the accuracies for estimating positive and negative feedback. Our results with a simulation experiment show that learning solely from predicted feedback based on facial expressions is possible.
  arXiv  Detail & Related papers  (2020-01-23T17:50:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.