ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation

• URL: http://arxiv.org/abs/2210.07450v1
• Date: Fri, 14 Oct 2022 01:32:15 GMT
• Title: ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation
• Authors: Noriaki Hirose, Dhruv Shah, Ajay Sridhar, Sergey Levine
• Abstract summary: We propose a novel framework, ExAug, to augment the experiences of different robot platforms from multiple datasets in diverse environments. The trained policy is evaluated on two new robot platforms with three different cameras in indoor and outdoor environments with obstacles.
• Score: 73.63212031963843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning techniques rely on large and diverse datasets for generalization. Computer vision, natural language processing, and other applications can often reuse public datasets to train many different models. However, due to differences in physical configurations, it is challenging to leverage public datasets for training robotic control policies on new robot platforms or for new tasks. In this work, we propose a novel framework, ExAug to augment the experiences of different robot platforms from multiple datasets in diverse environments. ExAug leverages a simple principle: by extracting 3D information in the form of a point cloud, we can create much more complex and structured augmentations, utilizing both generating synthetic images and geometric-aware penalization that would have been suitable in the same situation for a different robot, with different size, turning radius, and camera placement. The trained policy is evaluated on two new robot platforms with three different cameras in indoor and outdoor environments with obstacles.

Related papers

• Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments [26.66666135624716]
  We present Robot Utility Models (RUMs), a framework for training and deploying zero-shot robot policies. RUMs can generalize to new environments without any finetuning. We train five utility models for opening cabinet doors, opening drawers, picking up napkins, picking up paper bags, and reorienting fallen objects.
  arXiv  Detail & Related papers  (2024-09-09T17:59:50Z)
• Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation [49.03165169369552]
  By training a single policy across many different kinds of robots, a robot learning method can leverage much broader and more diverse datasets. We propose CrossFormer, a scalable and flexible transformer-based policy that can consume data from any embodiment. We demonstrate that the same network weights can control vastly different robots, including single and dual arm manipulation systems, wheeled robots, quadcopters, and quadrupeds.
  arXiv  Detail & Related papers  (2024-08-21T17:57:51Z)
• Octo: An Open-Source Generalist Robot Policy [88.14295917143188]
  We introduce Octo, a large transformer-based policy trained on 800k trajectories from the Open X-Embodiment dataset. It can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPU. We also perform detailed ablations of design decisions for the Octo model, from architecture to training data, to guide future research on building generalist robot models.
  arXiv  Detail & Related papers  (2024-05-20T17:57:01Z)
• Polybot: Training One Policy Across Robots While Embracing Variability [70.74462430582163]
  We propose a set of key design decisions to train a single policy for deployment on multiple robotic platforms. Our framework first aligns the observation and action spaces of our policy across embodiments via utilizing wrist cameras. We evaluate our method on a dataset collected over 60 hours spanning 6 tasks and 3 robots with varying joint configurations and sizes.
  arXiv  Detail & Related papers  (2023-07-07T17:21:16Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• Scaling Robot Learning with Semantically Imagined Experience [21.361979238427722]
  Recent advances in robot learning have shown promise in enabling robots to perform manipulation tasks. One of the key contributing factors to this progress is the scale of robot data used to train the models. We propose an alternative route and leverage text-to-image foundation models widely used in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-02-22T18:47:51Z)
• GNM: A General Navigation Model to Drive Any Robot [67.40225397212717]
  General goal-conditioned model for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots. We analyze the necessary design decisions for effective data sharing across robots. We deploy the trained GNM on a range of new robots, including an under quadrotor.
  arXiv  Detail & Related papers  (2022-10-07T07:26:41Z)
• Single-view robot pose and joint angle estimation via render & compare [40.05546237998603]
  We introduce RoboPose, a method to estimate the joint angles and the 6D camera-to-robot pose of a known articulated robot from a single RGB image. This is an important problem to grant mobile and itinerant autonomous systems the ability to interact with other robots.
  arXiv  Detail & Related papers  (2021-04-19T14:48:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.