Learning Latent Representations to Co-Adapt to Humans

• URL: http://arxiv.org/abs/2212.09586v3
• Date: Sat, 19 Aug 2023 23:47:28 GMT
• Title: Learning Latent Representations to Co-Adapt to Humans
• Authors: Sagar Parekh, Dylan P. Losey
• Abstract summary: Non-stationary humans are challenging for robot learners. In this paper we introduce an algorithmic formalism that enables robots to co-adapt alongside dynamic humans.
• Score: 12.71953776723672
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When robots interact with humans in homes, roads, or factories the human's behavior often changes in response to the robot. Non-stationary humans are challenging for robot learners: actions the robot has learned to coordinate with the original human may fail after the human adapts to the robot. In this paper we introduce an algorithmic formalism that enables robots (i.e., ego agents) to co-adapt alongside dynamic humans (i.e., other agents) using only the robot's low-level states, actions, and rewards. A core challenge is that humans not only react to the robot's behavior, but the way in which humans react inevitably changes both over time and between users. To deal with this challenge, our insight is that -- instead of building an exact model of the human -- robots can learn and reason over high-level representations of the human's policy and policy dynamics. Applying this insight we develop RILI: Robustly Influencing Latent Intent. RILI first embeds low-level robot observations into predictions of the human's latent strategy and strategy dynamics. Next, RILI harnesses these predictions to select actions that influence the adaptive human towards advantageous, high reward behaviors over repeated interactions. We demonstrate that -- given RILI's measured performance with users sampled from an underlying distribution -- we can probabilistically bound RILI's expected performance across new humans sampled from the same distribution. Our simulated experiments compare RILI to state-of-the-art representation and reinforcement learning baselines, and show that RILI better learns to coordinate with imperfect, noisy, and time-varying agents. Finally, we conduct two user studies where RILI co-adapts alongside actual humans in a game of tag and a tower-building task. See videos of our user studies here: https://youtu.be/WYGO5amDXbQ

Related papers

• HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation [50.616995671367704]
  We present a high-dimensional, simulated robot learning benchmark, HumanoidBench, featuring a humanoid robot equipped with dexterous hands. Our findings reveal that state-of-the-art reinforcement learning algorithms struggle with most tasks, whereas a hierarchical learning approach achieves superior performance when supported by robust low-level policies.
  arXiv  Detail & Related papers  (2024-03-15T17:45:44Z)
• Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots [119.55240471433302]
  Habitat 3.0 is a simulation platform for studying collaborative human-robot tasks in home environments. It addresses challenges in modeling complex deformable bodies and diversity in appearance and motion. Human-in-the-loop infrastructure enables real human interaction with simulated robots via mouse/keyboard or a VR interface.
  arXiv  Detail & Related papers  (2023-10-19T17:29:17Z)
• SACSoN: Scalable Autonomous Control for Social Navigation [62.59274275261392]
  We develop methods for training policies for socially unobtrusive navigation. By minimizing this counterfactual perturbation, we can induce robots to behave in ways that do not alter the natural behavior of humans in the shared space. We collect a large dataset where an indoor mobile robot interacts with human bystanders.
  arXiv  Detail & Related papers  (2023-06-02T19:07:52Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration [57.045140028275036]
  We show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning. We propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy.
  arXiv  Detail & Related papers  (2022-12-08T15:56:13Z)
• Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
  We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
  arXiv  Detail & Related papers  (2022-11-16T16:26:48Z)
• Learning Preferences for Interactive Autonomy [1.90365714903665]
  This thesis is an attempt towards learning reward functions from human users by using other, more reliable data modalities. We first propose various forms of comparative feedback, e.g., pairwise comparisons, best-of-many choices, rankings, scaled comparisons; and describe how a robot can use these various forms of human feedback to infer a reward function.
  arXiv  Detail & Related papers  (2022-10-19T21:34:51Z)
• RILI: Robustly Influencing Latent Intent [7.025418443146435]
  We propose a robust approach that learns to influence changing partner dynamics. Our method first trains with a set of partners across repeated interactions. We then rapidly adapt to new partners by sampling trajectories the robot learned with the original partners.
  arXiv  Detail & Related papers  (2022-03-23T19:55:49Z)
• Reasoning about Counterfactuals to Improve Human Inverse Reinforcement Learning [5.072077366588174]
  Humans naturally infer other agents' beliefs and desires by reasoning about their observable behavior. We propose to incorporate the learner's current understanding of the robot's decision making into our model of human IRL. We also propose a novel measure for estimating the difficulty for a human to predict instances of a robot's behavior in unseen environments.
  arXiv  Detail & Related papers  (2022-03-03T17:06:37Z)
• Physical Interaction as Communication: Learning Robot Objectives Online from Human Corrections [33.807697939765205]
  Physical human-robot interaction (pHRI) is often intentional -- the human intervenes on purpose because the robot is not doing the task correctly. In this paper, we argue that when pHRI is intentional it is also informative: the robot can leverage interactions to learn how it should complete the rest of its current task even after the human lets go. Our results indicate that learning from pHRI leads to better task performance and improved human satisfaction.
  arXiv  Detail & Related papers  (2021-07-06T02:25:39Z)
• Dynamically Switching Human Prediction Models for Efficient Planning [32.180808286226075]
  We give the robot access to a suite of human models and enable it to assess the performance-computation trade-off online. Our experiments in a driving simulator showcase how the robot can achieve performance comparable to always using the best human model.
  arXiv  Detail & Related papers  (2021-03-13T23:48:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.