ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks

• URL: http://arxiv.org/abs/2412.13211v2
• Date: Fri, 20 Dec 2024 05:21:39 GMT
• Title: ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks
• Authors: Arth Shukla, Stone Tao, Hao Su,
• Abstract summary: MS-HAB is a holistic benchmark for low-level manipulation and in-home object rearrangement. We support realistic low-level control and achieve over 3x the speed of previous magical grasp implementations at similar GPU memory usage.
• Score: 18.672482188560622
• License:
• Abstract: High-quality benchmarks are the foundation for embodied AI research, enabling significant advancements in long-horizon navigation, manipulation and rearrangement tasks. However, as frontier tasks in robotics get more advanced, they require faster simulation speed, more intricate test environments, and larger demonstration datasets. To this end, we present MS-HAB, a holistic benchmark for low-level manipulation and in-home object rearrangement. First, we provide a GPU-accelerated implementation of the Home Assistant Benchmark (HAB). We support realistic low-level control and achieve over 3x the speed of previous magical grasp implementations at similar GPU memory usage. Second, we train extensive reinforcement learning (RL) and imitation learning (IL) baselines for future work to compare against. Finally, we develop a rule-based trajectory filtering system to sample specific demonstrations from our RL policies which match predefined criteria for robot behavior and safety. Combining demonstration filtering with our fast environments enables efficient, controlled data generation at scale.

Related papers

• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Accelerating Point Cloud Ground Segmentation: From Mechanical to Solid-State Lidars [6.0753266069240235]
  We first benchmark point-based, grid-based, and range image-based ground segmentation algorithms. Our results indicate that the range image-based method offers superior performance and robustness. Implementing the proposed algorithm on an FPGA demonstrates significant improvements in processing speed and scalability of resource usage.
  arXiv  Detail & Related papers  (2024-08-19T20:39:21Z)
• PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning [13.564676246832544]
  We introduce PLANRL, a framework that chooses when the robot should use classical motion planning and when it should learn a policy. PLANRL switches between two modes of operation: reaching a waypoint using classical techniques when away from the objects and fine-grained manipulation control when about to interact with objects. We evaluate our approach across multiple challenging simulation environments and real-world tasks, demonstrating superior performance in terms of adaptability, efficiency, and generalization compared to existing methods.
  arXiv  Detail & Related papers  (2024-08-07T19:30:08Z)
• Re-Evaluating LiDAR Scene Flow for Autonomous Driving [80.37947791534985]
  Popular benchmarks for self-supervised LiDAR scene flow have unrealistic rates of dynamic motion, unrealistic correspondences, and unrealistic sampling patterns. We evaluate a suite of top methods on a suite of real-world datasets. We show that despite the emphasis placed on learning, most performance gains are caused by pre- and post-processing steps.
  arXiv  Detail & Related papers  (2023-04-04T22:45:50Z)
• ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills [24.150758623016195]
  We present ManiSkill2, the next generation of the SAPIEN ManiSkill benchmark for generalizable manipulation skills. ManiSkill2 includes 20 manipulation task families with 2000+ object models and 4M+ demonstration frames. It defines a unified interface and evaluation protocol to support a wide range of algorithms. It empowers fast visual input learning algorithms so that a CNN-based policy can collect samples at about 2000 FPS.
  arXiv  Detail & Related papers  (2023-02-09T14:24:01Z)
• Multitask Adaptation by Retrospective Exploration with Learned World Models [77.34726150561087]
  We propose a meta-learned addressing model called RAMa that provides training samples for the MBRL agent taken from task-agnostic storage. The model is trained to maximize the expected agent's performance by selecting promising trajectories solving prior tasks from the storage.
  arXiv  Detail & Related papers  (2021-10-25T20:02:57Z)
• Habitat 2.0: Training Home Assistants to Rearrange their Habitat [122.54624752876276]
  We introduce Habitat 2.0 (H2.0), a simulation platform for training virtual robots in interactive 3D environments. We make contributions to all levels of the embodied AI stack - data, simulation, and benchmark tasks.
  arXiv  Detail & Related papers  (2021-06-28T05:42:15Z)
• PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning [84.30765628008207]
  We propose a novel method, dubbed PlayVirtual, which augments cycle-consistent virtual trajectories to enhance the data efficiency for RL feature representation learning. Our method outperforms the current state-of-the-art methods by a large margin on both benchmarks.
  arXiv  Detail & Related papers  (2021-06-08T07:37:37Z)
• Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks with Base Controllers [26.807673929816026]
  We propose a method of learning long-horizon sparse-reward tasks utilizing one or more traditional base controllers. Our algorithm incorporates the existing base controllers into stages of exploration, value learning, and policy update. Our method bears the potential of leveraging existing industrial robot manipulation systems to build more flexible and intelligent controllers.
  arXiv  Detail & Related papers  (2020-11-24T14:23:57Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)
• AIBench Training: Balanced Industry-Standard AI Training Benchmarking [26.820244556465333]
  Earlier-stage evaluations of a new AI architecture/system need affordable benchmarks. We use real-world benchmarks to cover the factors space that impacts the learning dynamics. We contribute by far the most comprehensive AI training benchmark suite.
  arXiv  Detail & Related papers  (2020-04-30T11:08:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.