Efficient Robotic Object Search via HIEM: Hierarchical Policy Learning with Intrinsic-Extrinsic Modeling

• URL: http://arxiv.org/abs/2010.08596v2
• Date: Tue, 2 Mar 2021 19:45:02 GMT
• Title: Efficient Robotic Object Search via HIEM: Hierarchical Policy Learning with Intrinsic-Extrinsic Modeling
• Authors: Xin Ye and Yezhou Yang
• Abstract summary: We present a novel policy learning paradigm for the object search task, based on hierarchical and interpretable modeling with an intrinsic-extrinsic reward setting. Experiments conducted on the House3D environment validate and show that the robot, trained with our model, can perform the object search task in a more optimal and interpretable way.
• Score: 33.89793938441333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the significant success at enabling robots with autonomous behaviors makes deep reinforcement learning a promising approach for robotic object search task, the deep reinforcement learning approach severely suffers from the nature sparse reward setting of the task. To tackle this challenge, we present a novel policy learning paradigm for the object search task, based on hierarchical and interpretable modeling with an intrinsic-extrinsic reward setting. More specifically, we explore the environment efficiently through a proxy low-level policy which is driven by the intrinsic rewarding sub-goals. We further learn our hierarchical policy from the efficient exploration experience where we optimize both of our high-level and low-level policies towards the extrinsic rewarding goal to perform the object search task well. Experiments conducted on the House3D environment validate and show that the robot, trained with our model, can perform the object search task in a more optimal and interpretable way.

Related papers

• Cognitive Planning for Object Goal Navigation using Generative AI Models [0.979851640406258]
  We present a novel framework for solving the object goal navigation problem that generates efficient exploration strategies. Our approach enables a robot to navigate unfamiliar environments by leveraging Large Language Models (LLMs) and Large Vision-Language Models (LVLMs)
  arXiv  Detail & Related papers  (2024-03-30T10:54:59Z)
• Enhancing Robotic Navigation: An Evaluation of Single and Multi-Objective Reinforcement Learning Strategies [0.9208007322096532]
  This study presents a comparative analysis between single-objective and multi-objective reinforcement learning methods for training a robot to navigate effectively to an end goal. By modifying the reward function to return a vector of rewards, each pertaining to a distinct objective, the robot learns a policy that effectively balances the different goals.
  arXiv  Detail & Related papers  (2023-12-13T08:00:26Z)
• Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation [10.21450780640562]
  We introduce a novel interactive multi-object search task in which a robot has to open doors to navigate rooms and search inside cabinets and drawers to find target objects. These new challenges require combining manipulation and navigation skills in unexplored environments. We present HIMOS, a hierarchical reinforcement learning approach that learns to compose exploration, navigation, and manipulation skills.
  arXiv  Detail & Related papers  (2023-07-12T12:25:33Z)
• Deep Hierarchical Planning from Pixels [86.14687388689204]
  Director is a method for learning hierarchical behaviors directly from pixels by planning inside the latent space of a learned world model. Despite operating in latent space, the decisions are interpretable because the world model can decode goals into images for visualization. Director also learns successful behaviors across a wide range of environments, including visual control, Atari games, and DMLab levels.
  arXiv  Detail & Related papers  (2022-06-08T18:20:15Z)
• Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space [76.46113138484947]
  General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach goals for a wide range of tasks on command. We propose Planning to Practice, a method that makes it practical to train goal-conditioned policies for long-horizon tasks.
  arXiv  Detail & Related papers  (2022-05-17T06:58:17Z)
• Affordance Learning from Play for Sample-Efficient Policy Learning [30.701546777177555]
  We use a self-supervised visual affordance model from human teleoperated play data to enable efficient policy learning and motion planning. We combine model-based planning with model-free deep reinforcement learning to learn policies that favor the same object regions favored by people. We find that our policies train 4x faster than the baselines and generalize better to novel objects because our visual affordance model can anticipate their affordance regions.
  arXiv  Detail & Related papers  (2022-03-01T11:00:35Z)
• Landmark Policy Optimization for Object Navigation Task [77.34726150561087]
  This work studies object goal navigation task, which involves navigating to the closest object related to the given semantic category in unseen environments. Recent works have shown significant achievements both in the end-to-end Reinforcement Learning approach and modular systems, but need a big step forward to be robust and optimal. We propose a hierarchical method that incorporates standard task formulation and additional area knowledge as landmarks, with a way to extract these landmarks.
  arXiv  Detail & Related papers  (2021-09-17T12:28:46Z)
• Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment [73.9469267445146]
  First-person object-interaction tasks in high-fidelity, 3D, simulated environments such as the AI2Thor pose significant sample-efficiency challenges for reinforcement learning agents. We show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task.
  arXiv  Detail & Related papers  (2020-10-28T19:27:26Z)
• Follow the Object: Curriculum Learning for Manipulation Tasks with Imagined Goals [8.98526174345299]
  This paper introduces a notion of imaginary object goals. For a given manipulation task, the object of interest is first trained to reach a desired target position on its own. The object policy is then leveraged to build a predictive model of plausible object trajectories. The proposed algorithm, Follow the Object, has been evaluated on 7 MuJoCo environments.
  arXiv  Detail & Related papers  (2020-08-05T12:19:14Z)
• Learning with AMIGo: Adversarially Motivated Intrinsic Goals [63.680207855344875]
  AMIGo is a goal-generating teacher that proposes Adversarially Motivated Intrinsic Goals. We show that our method generates a natural curriculum of self-proposed goals which ultimately allows the agent to solve challenging procedurally-generated tasks.
  arXiv  Detail & Related papers  (2020-06-22T10:22:08Z)
• Automatic Curriculum Learning through Value Disagreement [95.19299356298876]
  Continually solving new, unsolved tasks is the key to learning diverse behaviors. In the multi-task domain, where an agent needs to reach multiple goals, the choice of training goals can largely affect sample efficiency. We propose setting up an automatic curriculum for goals that the agent needs to solve. We evaluate our method across 13 multi-goal robotic tasks and 5 navigation tasks, and demonstrate performance gains over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-17T03:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.