Hierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Search

• URL: http://arxiv.org/abs/2109.08973v1
• Date: Sat, 18 Sep 2021 17:24:37 GMT
• Title: Hierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Search
• Authors: Fan Bai, Fei Meng, Jianbang Liu, Jiankun Wang, Max Q.-H. Meng
• Abstract summary: We propose a hierarchical policy to divide and conquer for non-prehensile multi-object rearrangement. In the high-level policy, the Monte Carlo Tree Search efficiently searches for the optimal rearrangement sequence among multiple objects. In the low-level policy, the robot plans the paths according to the order of path primitives and manipulates the objects to approach the goal poses one by one.
• Score: 30.31462739429364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Non-prehensile multi-object rearrangement is a robotic task of planning feasible paths and transferring multiple objects to their predefined target poses without grasping. It needs to consider how each object reaches the target and the order of object movement, which significantly deepens the complexity of the problem. To address these challenges, we propose a hierarchical policy to divide and conquer for non-prehensile multi-object rearrangement. In the high-level policy, guided by a designed policy network, the Monte Carlo Tree Search efficiently searches for the optimal rearrangement sequence among multiple objects, which benefits from imitation and reinforcement. In the low-level policy, the robot plans the paths according to the order of path primitives and manipulates the objects to approach the goal poses one by one. We verify through experiments that the proposed method can achieve a higher success rate, fewer steps, and shorter path length compared with the state-of-the-art.

Related papers

• MANER: Multi-Agent Neural Rearrangement Planning of Objects in Cluttered Environments [8.15681999722805]
  This paper proposes a learning-based framework for multi-agent object rearrangement planning. It addresses the challenges of task sequencing and path planning in complex environments.
  arXiv  Detail & Related papers  (2023-06-10T23:53:28Z)
• Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generation [41.79944184861954]
  Goal-oriented Script Generation is a new task of generating a list of steps that can fulfill the given goal. In this paper, we propose to extend the task from the perspective of cognitive theory.
  arXiv  Detail & Related papers  (2023-05-18T12:10:06Z)
• Imitating Graph-Based Planning with Goal-Conditioned Policies [72.61631088613048]
  We present a self-imitation scheme which distills a subgoal-conditioned policy into the target-goal-conditioned policy. We empirically show that our method can significantly boost the sample-efficiency of the existing goal-conditioned RL methods.
  arXiv  Detail & Related papers  (2023-03-20T14:51:10Z)
• Effective Baselines for Multiple Object Rearrangement Planning in Partially Observable Mapped Environments [5.32429768581469]
  This paper aims to enable home-assistive intelligent agents to efficiently plan for rearrangement under partial observability. We investigate monolithic and modular deep reinforcement learning (DRL) methods for planning in our setting. We find that monolithic DRL methods do not succeed at long-horizon planning needed for multi-object rearrangement. We also show that our greedy modular agents are empirically optimal when the objects that need to be rearranged are uniformly distributed in the environment.
  arXiv  Detail & Related papers  (2023-01-24T08:03:34Z)
• Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport [83.06265788137443]
  We address key challenges in long-horizon embodied exploration and navigation by proposing a new object transport task and a novel modular framework for temporally extended navigation. Our first contribution is the design of a novel Long-HOT environment focused on deep exploration and long-horizon planning. We propose a modular hierarchical transport policy (HTP) that builds a topological graph of the scene to perform exploration with the help of weighted frontiers.
  arXiv  Detail & Related papers  (2022-10-28T05:30:49Z)
• A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search [71.14527779661181]
  Visual room rearrangement evaluates an agent's ability to rearrange objects based solely on visual input. We propose a simple yet effective method for this problem: (1) search for and map which objects need to be rearranged, and (2) rearrange each object until the task is complete.
  arXiv  Detail & Related papers  (2022-06-21T02:33:57Z)
• Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space [76.46113138484947]
  General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach goals for a wide range of tasks on command. We propose Planning to Practice, a method that makes it practical to train goal-conditioned policies for long-horizon tasks.
  arXiv  Detail & Related papers  (2022-05-17T06:58:17Z)
• NeRP: Neural Rearrangement Planning for Unknown Objects [49.191284597526]
  We propose NeRP (Neural Rearrangement Planning), a deep learning based approach for multi-step neural object rearrangement planning. NeRP works with never-before-seen objects, that is trained on simulation data, and generalizes to the real world.
  arXiv  Detail & Related papers  (2021-06-02T17:56:27Z)
• Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning [78.65083326918351]
  We consider alternatives to an implicit sequential planning assumption. We propose Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS) for approximating the optimal plan. We show that this algorithmic flexibility over planning order leads to improved results in navigation tasks in grid-worlds.
  arXiv  Detail & Related papers  (2020-04-23T18:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.