Privileged Sensing Scaffolds Reinforcement Learning

• URL: http://arxiv.org/abs/2405.14853v1
• Date: Thu, 23 May 2024 17:57:14 GMT
• Title: Privileged Sensing Scaffolds Reinforcement Learning
• Authors: Edward S. Hu, James Springer, Oleh Rybkin, Dinesh Jayaraman,
• Abstract summary: We consider sensory scaffolding setups for training artificial agents. "Scaffolder" is a reinforcement learning approach which effectively exploits privileged sensing in critics. Agents must use privileged camera sensing to train blind hurdlers.
• Score: 28.100745092661587
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We need to look at our shoelaces as we first learn to tie them but having mastered this skill, can do it from touch alone. We call this phenomenon "sensory scaffolding": observation streams that are not needed by a master might yet aid a novice learner. We consider such sensory scaffolding setups for training artificial agents. For example, a robot arm may need to be deployed with just a low-cost, robust, general-purpose camera; yet its performance may improve by having privileged training-time-only access to informative albeit expensive and unwieldy motion capture rigs or fragile tactile sensors. For these settings, we propose "Scaffolder", a reinforcement learning approach which effectively exploits privileged sensing in critics, world models, reward estimators, and other such auxiliary components that are only used at training time, to improve the target policy. For evaluating sensory scaffolding agents, we design a new "S3" suite of ten diverse simulated robotic tasks that explore a wide range of practical sensor setups. Agents must use privileged camera sensing to train blind hurdlers, privileged active visual perception to help robot arms overcome visual occlusions, privileged touch sensors to train robot hands, and more. Scaffolder easily outperforms relevant prior baselines and frequently performs comparably even to policies that have test-time access to the privileged sensors. Website: https://penn-pal-lab.github.io/scaffolder/

Related papers

• Robust Robot Walker: Learning Agile Locomotion over Tiny Traps [28.920959351960413]
  We propose a novel approach that enables quadruped robots to pass various small obstacles, or "tiny traps" Existing methods often rely on exteroceptive sensors, which can be unreliable for detecting such tiny traps. We introduce a two-stage training framework incorporating a contact encoder and a classification head to learn implicit representations of different traps.
  arXiv  Detail & Related papers  (2024-09-11T16:50:29Z)
• DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexterity [11.450027373581019]
  We introduce a multi-finger robot system designed to manipulate objects using the sense of touch, without relying on vision. For tasks that mimic daily life, the robot uses its sense of touch to manipulate randomly placed objects in dark.
  arXiv  Detail & Related papers  (2024-01-23T05:37:32Z)
• See to Touch: Learning Tactile Dexterity through Visual Incentives [20.586023376454115]
  We present Tactile Adaptation from Visual Incentives (TAVI), a new framework that enhances tactile-based dexterity. On six challenging tasks, TAVI achieves a success rate of 73% using our four-fingered Allegro robot hand.
  arXiv  Detail & Related papers  (2023-09-21T17:58:13Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Rotating without Seeing: Towards In-hand Dexterity through Touch [43.87509744768282]
  We present Touch Dexterity, a new system that can perform in-hand object rotation using only touching without seeing the object. Instead of relying on precise tactile sensing in a small region, we introduce a new system design using dense binary force sensors (touch or no touch) overlaying one side of the whole robot hand. We train an in-hand rotation policy using Reinforcement Learning on diverse objects in simulation. Relying on touch-only sensing, we can directly deploy the policy in a real robot hand and rotate novel objects that are not presented in training.
  arXiv  Detail & Related papers  (2023-03-20T05:38:30Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
  We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
  arXiv  Detail & Related papers  (2022-11-16T16:26:48Z)
• DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video [86.49357517864937]
  We propose DexVIP, an approach to learn dexterous robotic grasping from human-object interaction videos. We do this by curating grasp images from human-object interaction videos and imposing a prior over the agent's hand pose. We demonstrate that DexVIP compares favorably to existing approaches that lack a hand pose prior or rely on specialized tele-operation equipment.
  arXiv  Detail & Related papers  (2022-02-01T00:45:57Z)
• Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills [93.12417203541948]
  We propose the objective of learning a functional understanding of the environment by learning to reach any goal state in a given dataset. We find that our method can operate on high-dimensional camera images and learn a variety of skills on real robots that generalize to previously unseen scenes and objects.
  arXiv  Detail & Related papers  (2021-04-15T20:10:11Z)
• Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
  Dexterous robotic hands are appealing for their agility and human-like morphology. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
  arXiv  Detail & Related papers  (2020-09-03T04:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.