Predicting Stable Configurations for Semantic Placement of Novel Objects

• URL: http://arxiv.org/abs/2108.12062v1
• Date: Thu, 26 Aug 2021 23:05:05 GMT
• Title: Predicting Stable Configurations for Semantic Placement of Novel Objects
• Authors: Chris Paxton, Chris Xie, Tucker Hermans, and Dieter Fox
• Abstract summary: Our goal is to enable robots to repose previously unseen objects according to learned semantic relationships in novel environments. We build our models and training from the ground up to be tightly integrated with our proposed planning algorithm for semantic placement of unknown objects. Our approach enables motion planning for semantic rearrangement of unknown objects in scenes with varying geometry from only RGB-D sensing.
• Score: 37.18437299513799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human environments contain numerous objects configured in a variety of arrangements. Our goal is to enable robots to repose previously unseen objects according to learned semantic relationships in novel environments. We break this problem down into two parts: (1) finding physically valid locations for the objects and (2) determining if those poses satisfy learned, high-level semantic relationships. We build our models and training from the ground up to be tightly integrated with our proposed planning algorithm for semantic placement of unknown objects. We train our models purely in simulation, with no fine-tuning needed for use in the real world. Our approach enables motion planning for semantic rearrangement of unknown objects in scenes with varying geometry from only RGB-D sensing. Our experiments through a set of simulated ablations demonstrate that using a relational classifier alone is not sufficient for reliable planning. We further demonstrate the ability of our planner to generate and execute diverse manipulation plans through a set of real-world experiments with a variety of objects.

Related papers

• ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decomposition [8.654140442734354]
  Task-oriented grasping of unfamiliar objects is a necessary skill for robots in dynamic in-home environments. We present a novel zero-shot task-oriented grasping method leveraging a geometric decomposition of the target object into simple convex shapes. Our approach employs minimal essential information - the object's name and the intended task - to facilitate zero-shot task-oriented grasping.
  arXiv  Detail & Related papers  (2024-03-26T19:26:53Z)
• Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds [23.923838486208524]
  We investigate a variation of the 3D registration problem, named multi-model 3D registration. In the multi-model registration problem, we are given two point clouds picturing a set of objects at different poses. We want to simultaneously reconstruct how all objects moved between the two point clouds.
  arXiv  Detail & Related papers  (2024-02-16T18:01:43Z)
• ICGNet: A Unified Approach for Instance-Centric Grasping [42.92991092305974]
  We introduce an end-to-end architecture for object-centric grasping. We show the effectiveness of the proposed method by extensively evaluating it against state-of-the-art methods on synthetic datasets.
  arXiv  Detail & Related papers  (2024-01-18T12:41:41Z)
• Learning Extrinsic Dexterity with Parameterized Manipulation Primitives [8.7221770019454]
  We learn a sequence of actions that utilize the environment to change the object's pose. Our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace.
  arXiv  Detail & Related papers  (2023-10-26T21:28:23Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Planning for Complex Non-prehensile Manipulation Among Movable Objects by Interleaving Multi-Agent Pathfinding and Physics-Based Simulation [23.62057790524675]
  Real-world manipulation problems in heavy clutter require robots to reason about potential contacts with objects in the environment. We focus on pick-and-place style tasks to retrieve a target object from a shelf where some movable' objects must be rearranged in order to solve the task. In particular, our motivation is to allow the robot to reason over and consider non-prehensile rearrangement actions that lead to complex robot-object and object-object interactions.
  arXiv  Detail & Related papers  (2023-03-23T15:29:27Z)
• Discovering Objects that Can Move [55.743225595012966]
  We study the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. We choose to focus on dynamic objects -- entities that can move independently in the world.
  arXiv  Detail & Related papers  (2022-03-18T21:13:56Z)
• Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Grounding [93.82542533426766]
  We propose a Suspected Object Transformation mechanism (SOT) to encourage the target object selection among the suspected ones. SOT can be seamlessly integrated into existing CNN and Transformer-based one-stage visual grounders. Extensive experiments demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2022-03-10T06:41:07Z)
• TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction [57.1209039399599]
  We propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein. In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation.
  arXiv  Detail & Related papers  (2021-05-16T16:15:05Z)
• PackIt: A Virtual Environment for Geometric Planning [68.79816936618454]
  PackIt is a virtual environment to evaluate and potentially learn the ability to do geometric planning. We construct a set of challenging packing tasks using an evolutionary algorithm.
  arXiv  Detail & Related papers  (2020-07-21T22:51:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.