Touch-based Curiosity for Sparse-Reward Tasks

• URL: http://arxiv.org/abs/2104.00442v1
• Date: Thu, 1 Apr 2021 12:49:29 GMT
• Title: Touch-based Curiosity for Sparse-Reward Tasks
• Authors: Sai Rajeswar, Cyril Ibrahim, Nitin Surya, Florian Golemo, David Vazquez, Aaron Courville, Pedro O. Pinheiro
• Abstract summary: We use surprise from mismatches in touch feedback to guide exploration in hard sparse-reward reinforcement learning tasks. Our approach, Touch-based Curiosity (ToC), learns what visible objects interactions are supposed to "feel" like. We test our approach on a range of touch-intensive robot arm tasks.
• Score: 15.766198618516137
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robots in many real-world settings have access to force/torque sensors in their gripper and tactile sensing is often necessary in tasks that involve contact-rich motion. In this work, we leverage surprise from mismatches in touch feedback to guide exploration in hard sparse-reward reinforcement learning tasks. Our approach, Touch-based Curiosity (ToC), learns what visible objects interactions are supposed to "feel" like. We encourage exploration by rewarding interactions where the expectation and the experience don't match. In our proposed method, an initial task-independent exploration phase is followed by an on-task learning phase, in which the original interactions are relabeled with on-task rewards. We test our approach on a range of touch-intensive robot arm tasks (e.g. pushing objects, opening doors), which we also release as part of this work. Across multiple experiments in a simulated setting, we demonstrate that our method is able to learn these difficult tasks through sparse reward and curiosity alone. We compare our cross-modal approach to single-modality (touch- or vision-only) approaches as well as other curiosity-based methods and find that our method performs better and is more sample-efficient.

Related papers

• KOI: Accelerating Online Imitation Learning via Hybrid Key-state Guidance [51.09834120088799]
  We introduce the hybrid Key-state guided Online Imitation (KOI) learning method. We use visual-language models to extract semantic key states from expert trajectory, indicating the objectives of "what to do" Within the intervals between semantic key states, optical flow is employed to capture motion key states to understand the mechanisms of "how to do"
  arXiv  Detail & Related papers  (2024-08-06T02:53:55Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
  We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
  arXiv  Detail & Related papers  (2022-11-16T16:26:48Z)
• TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors [15.418884994244996]
  We focus on the process of guiding tactile exploration, and its interplay with task-related decision making. We propose TANDEM, an architecture to learn efficient exploration strategies in conjunction with decision making. We demonstrate this method on a tactile object recognition task, where a robot equipped with a touch sensor must explore and identify an object from a known set based on tactile feedback alone.
  arXiv  Detail & Related papers  (2022-03-01T23:55:09Z)
• Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation [55.31301153979621]
  We tackle real-world long-horizon robot manipulation tasks through skill discovery. We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations. Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks.
  arXiv  Detail & Related papers  (2021-09-28T16:18:54Z)
• Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration [20.38772636693469]
  We argue that merely using curiosity for fast environment exploration or as a bonus reward for a specific task does not harness the full potential of this technique. We propose to shift the focus towards retaining the behaviours which emerge during curiosity-based learning.
  arXiv  Detail & Related papers  (2021-09-17T15:28:25Z)
• Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation [40.81570120196115]
  In sparse goal settings, an agent does not receive any positive feedback until randomly achieving the goal. Inspired by touch-based exploration observed in children, we formulate an intrinsic reward based on the sum of forces between a robot's force sensors and manipulation objects. We show that our solution accelerates the exploration and outperforms state-of-the-art methods on three fundamental robot manipulation benchmarks.
  arXiv  Detail & Related papers  (2021-02-22T14:21:30Z)
• ACDER: Augmented Curiosity-Driven Experience Replay [16.755555854030412]
  We propose a novel method called Augmented Curiosity-Driven Experience Replay (ACDER) ACDER uses a new goal-oriented curiosity-driven exploration to encourage the agent to pursue novel and task-relevant states more purposefully. Experiments conducted on four challenging robotic manipulation tasks with binary rewards, including Reach, Push, Pick&Place and Multi-step Push.
  arXiv  Detail & Related papers  (2020-11-16T15:27:15Z)
• Hierarchical Affordance Discovery using Intrinsic Motivation [69.9674326582747]
  We propose an algorithm using intrinsic motivation to guide the learning of affordances for a mobile robot. This algorithm is capable to autonomously discover, learn and adapt interrelated affordances without pre-programmed actions. Once learned, these affordances may be used by the algorithm to plan sequences of actions in order to perform tasks of various difficulties.
  arXiv  Detail & Related papers  (2020-09-23T07:18:21Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.