Occlusion-Aware Search for Object Retrieval in Clutter

• URL: http://arxiv.org/abs/2011.03334v4
• Date: Tue, 31 Aug 2021 14:28:07 GMT
• Title: Occlusion-Aware Search for Object Retrieval in Clutter
• Authors: Wissam Bejjani, Wisdom C. Agboh, Mehmet R. Dogar and Matteo Leonetti
• Abstract summary: We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. We present a data-driven hybrid planner for generating occlusion-aware actions in closed-loop.
• Score: 4.693170687870612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven hybrid planner for generating occlusion-aware actions in closed-loop. The hybrid planner explores likely locations of the occluded target object as predicted by a learned distribution from the observation stream. The search is guided by a heuristic trained with reinforcement learning to act on observations with occlusions. We evaluate our approach in different simulation and real-world settings (video available on https://youtu.be/dY7YQ3LUVQg). The results validate that our approach can search and retrieve a target object in near real time in the real world while only being trained in simulation.

Related papers

• SeMoLi: What Moves Together Belongs Together [51.72754014130369]
  We tackle semi-supervised object detection based on motion cues. Recent results suggest that motion-based clustering methods can be used to pseudo-label instances of moving objects. We re-think this approach and suggest that both, object detection, as well as motion-inspired pseudo-labeling, can be tackled in a data-driven manner.
  arXiv  Detail & Related papers  (2024-02-29T18:54:53Z)
• Generalized Object Search [0.9137554315375919]
  This thesis develops methods and systems for (multi-)object search in 3D environments under uncertainty. I implement a robot-independent, environment-agnostic system for generalized object search in 3D. I deploy it on the Boston Dynamics Spot robot, the Kinova MOVO robot, and the Universal Robots UR5e robotic arm.
  arXiv  Detail & Related papers  (2023-01-24T16:41:36Z)
• H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactions [62.510951695174604]
  "Hypothesize, Simulate, Act, Update, and Repeat" (H-SAUR) is a probabilistic generative framework that generates hypotheses about how objects articulate given input observations. We show that the proposed model significantly outperforms the current state-of-the-art articulated object manipulation framework. We further improve the test-time efficiency of H-SAUR by integrating a learned prior from learning-based vision models.
  arXiv  Detail & Related papers  (2022-10-22T18:39:33Z)
• Object Manipulation via Visual Target Localization [64.05939029132394]
  Training agents to manipulate objects, poses many challenges. We propose an approach that explores the environment in search for target objects, computes their 3D coordinates once they are located, and then continues to estimate their 3D locations even when the objects are not visible. Our evaluations show a massive 3x improvement in success rate over a model that has access to the same sensory suite.
  arXiv  Detail & Related papers  (2022-03-15T17:59:01Z)
• SafePicking: Learning Safe Object Extraction via Object-Level Mapping [19.502587411252946]
  We present a system, SafePicking, that integrates object-level mapping and learning-based motion planning. Planning is done by learning a deep Q-network that receives observations of predicted poses and a depth-based heightmap to output a motion trajectory. Our results show that the observation fusion of poses and depth-sensing gives both better performance and robustness to the model.
  arXiv  Detail & Related papers  (2022-02-11T18:55:10Z)
• INVIGORATE: Interactive Visual Grounding and Grasping in Clutter [56.00554240240515]
  INVIGORATE is a robot system that interacts with human through natural language and grasps a specified object in clutter. We train separate neural networks for object detection, for visual grounding, for question generation, and for OBR detection and grasping. We build a partially observable Markov decision process (POMDP) that integrates the learned neural network modules.
  arXiv  Detail & Related papers  (2021-08-25T07:35:21Z)
• Learning to Track with Object Permanence [61.36492084090744]
  We introduce an end-to-end trainable approach for joint object detection and tracking. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
  arXiv  Detail & Related papers  (2021-03-26T04:43:04Z)
• Batch Exploration with Examples for Scalable Robotic Reinforcement Learning [63.552788688544254]
  Batch Exploration with Examples (BEE) explores relevant regions of the state-space guided by a modest number of human provided images of important states. BEE is able to tackle challenging vision-based manipulation tasks both in simulation and on a real Franka robot.
  arXiv  Detail & Related papers  (2020-10-22T17:49:25Z)
• Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter [43.668395529368354]
  We present a novel Deep RL procedure that combines teacher-aided exploration, ii) a critic with privileged information, andiii) mid-level representations. Our approach trains faster and converges to more efficient uncovering solutions than baselines and ablations, and that our uncovering policies lead to an average improvement in the graspability of the target object.
  arXiv  Detail & Related papers  (2020-08-13T18:23:00Z)
• Follow the Object: Curriculum Learning for Manipulation Tasks with Imagined Goals [8.98526174345299]
  This paper introduces a notion of imaginary object goals. For a given manipulation task, the object of interest is first trained to reach a desired target position on its own. The object policy is then leveraged to build a predictive model of plausible object trajectories. The proposed algorithm, Follow the Object, has been evaluated on 7 MuJoCo environments.
  arXiv  Detail & Related papers  (2020-08-05T12:19:14Z)
• Semantic Linking Maps for Active Visual Object Search [14.573513188682183]
  We exploit background knowledge about common spatial relations between landmark and target objects. We propose an active visual object search strategy method through our introduction of the Semantic Linking Maps (SLiM) model. Based on SLiM, we describe a hybrid search strategy that selects the next best view pose for searching for the target object.
  arXiv  Detail & Related papers  (2020-06-18T18:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.