What Can I Do Here? Learning New Skills by Imagining Visual Affordances

• URL: http://arxiv.org/abs/2106.00671v1
• Date: Tue, 1 Jun 2021 17:58:02 GMT
• Title: What Can I Do Here? Learning New Skills by Imagining Visual Affordances
• Authors: Alexander Khazatsky, Ashvin Nair, Daniel Jing, Sergey Levine
• Abstract summary: We show how generative models of possible outcomes can allow a robot to learn visual representations of affordances. In effect, prior data is used to learn what kinds of outcomes may be possible, such that when the robot encounters an unfamiliar setting, it can sample potential outcomes from its model. We show that visuomotor affordance learning (VAL) can be used to train goal-conditioned policies that operate on raw image inputs.
• Score: 128.65223577406587
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A generalist robot equipped with learned skills must be able to perform many tasks in many different environments. However, zero-shot generalization to new settings is not always possible. When the robot encounters a new environment or object, it may need to finetune some of its previously learned skills to accommodate this change. But crucially, previously learned behaviors and models should still be suitable to accelerate this relearning. In this paper, we aim to study how generative models of possible outcomes can allow a robot to learn visual representations of affordances, so that the robot can sample potentially possible outcomes in new situations, and then further train its policy to achieve those outcomes. In effect, prior data is used to learn what kinds of outcomes may be possible, such that when the robot encounters an unfamiliar setting, it can sample potential outcomes from its model, attempt to reach them, and thereby update both its skills and its outcome model. This approach, visuomotor affordance learning (VAL), can be used to train goal-conditioned policies that operate on raw image inputs, and can rapidly learn to manipulate new objects via our proposed affordance-directed exploration scheme. We show that VAL can utilize prior data to solve real-world tasks such drawer opening, grasping, and placing objects in new scenes with only five minutes of online experience in the new scene.

Related papers

• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• Exploring Visual Pre-training for Robot Manipulation: Datasets, Models and Methods [14.780597545674157]
  We investigate the effects of visual pre-training strategies on robot manipulation tasks from three fundamental perspectives. We propose a visual pre-training scheme for robot manipulation termed Vi-PRoM, which combines self-supervised learning and supervised learning.
  arXiv  Detail & Related papers  (2023-08-07T14:24:52Z)
• Evaluating Continual Learning on a Home Robot [30.620205237707342]
  We show how continual learning methods can be adapted for use on a real, low-cost home robot. We propose SANER, a method for continuously learning a library of skills, and ABIP, as the backbone to support it.
  arXiv  Detail & Related papers  (2023-06-04T17:14:49Z)
• Scaling Robot Learning with Semantically Imagined Experience [21.361979238427722]
  Recent advances in robot learning have shown promise in enabling robots to perform manipulation tasks. One of the key contributing factors to this progress is the scale of robot data used to train the models. We propose an alternative route and leverage text-to-image foundation models widely used in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-02-22T18:47:51Z)
• Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
  We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
  arXiv  Detail & Related papers  (2022-11-16T16:26:48Z)
• Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills [93.12417203541948]
  We propose the objective of learning a functional understanding of the environment by learning to reach any goal state in a given dataset. We find that our method can operate on high-dimensional camera images and learn a variety of skills on real robots that generalize to previously unseen scenes and objects.
  arXiv  Detail & Related papers  (2021-04-15T20:10:11Z)
• COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning [78.13740204156858]
  We show that we can reuse prior data to extend new skills simply through dynamic programming. We demonstrate the effectiveness of our approach by chaining together several behaviors seen in prior datasets for solving a new task. We train our policies in an end-to-end fashion, mapping high-dimensional image observations to low-level robot control commands.
  arXiv  Detail & Related papers  (2020-10-27T17:57:29Z)
• Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning [109.77163932886413]
  We show how to adapt vision-based robotic manipulation policies to new variations by fine-tuning via off-policy reinforcement learning. This adaptation uses less than 0.2% of the data necessary to learn the task from scratch. We find that our approach of adapting pre-trained policies leads to substantial performance gains over the course of fine-tuning.
  arXiv  Detail & Related papers  (2020-04-21T17:57:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.