Manipulation-Oriented Object Perception in Clutter through Affordance Coordinate Frames

• URL: http://arxiv.org/abs/2010.08202v3
• Date: Wed, 2 Mar 2022 04:54:04 GMT
• Title: Manipulation-Oriented Object Perception in Clutter through Affordance Coordinate Frames
• Authors: Xiaotong Chen, Kaizhi Zheng, Zhen Zeng, Shreshtha Basu, James Cooney, Jana Pavlasek, Odest Chadwicke Jenkins
• Abstract summary: In this work, we combine the notions of affordance and category-level pose, and introduce the Affordance Coordinate Frame (ACF) With ACF, we represent each object class in terms of individual affordance parts and the compatibility between them, where each part is associated with a part category-level pose for robot manipulation. In our experiments, we demonstrate that ACF outperforms state-of-the-art methods for object detection, as well as category-level pose estimation for object parts.
• Score: 10.90648422740674
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In order to enable robust operation in unstructured environments, robots should be able to generalize manipulation actions to novel object instances. For example, to pour and serve a drink, a robot should be able to recognize novel containers which afford the task. Most importantly, robots should be able to manipulate these novel containers to fulfill the task. To achieve this, we aim to provide robust and generalized perception of object affordances and their associated manipulation poses for reliable manipulation. In this work, we combine the notions of affordance and category-level pose, and introduce the Affordance Coordinate Frame (ACF). With ACF, we represent each object class in terms of individual affordance parts and the compatibility between them, where each part is associated with a part category-level pose for robot manipulation. In our experiments, we demonstrate that ACF outperforms state-of-the-art methods for object detection, as well as category-level pose estimation for object parts. We further demonstrate the applicability of ACF to robot manipulation tasks through experiments in a simulated environment.

Related papers

• Keypoint Abstraction using Large Models for Object-Relative Imitation Learning [78.92043196054071]
  Generalization to novel object configurations and instances across diverse tasks and environments is a critical challenge in robotics. Keypoint-based representations have been proven effective as a succinct representation for essential object capturing features. We propose KALM, a framework that leverages large pre-trained vision-language models to automatically generate task-relevant and cross-instance consistent keypoints.
  arXiv  Detail & Related papers  (2024-10-30T17:37:31Z)
• RPMArt: Towards Robust Perception and Manipulation for Articulated Objects [56.73978941406907]
  We propose a framework towards Robust Perception and Manipulation for Articulated Objects ( RPMArt) RPMArt learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud. We introduce an articulation-aware classification scheme to enhance its ability for sim-to-real transfer.
  arXiv  Detail & Related papers  (2024-03-24T05:55:39Z)
• ManiPose: A Comprehensive Benchmark for Pose-aware Object Manipulation in Robotics [55.85916671269219]
  This paper introduces ManiPose, a pioneering benchmark designed to advance the study of pose-varying manipulation tasks. A comprehensive dataset features geometrically consistent and manipulation-oriented 6D pose labels for 2936 real-world scanned rigid objects and 100 articulated objects. Our benchmark demonstrates notable advancements in pose estimation, pose-aware manipulation, and real-robot skill transfer.
  arXiv  Detail & Related papers  (2024-03-20T07:48:32Z)
• Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMs [53.66070434419739]
  Generalizable articulated object manipulation is essential for home-assistant robots. We propose a kinematic-aware prompting framework that prompts Large Language Models with kinematic knowledge of objects to generate low-level motion waypoints. Our framework outperforms traditional methods on 8 categories seen and shows a powerful zero-shot capability for 8 unseen articulated object categories.
  arXiv  Detail & Related papers  (2023-11-06T03:26:41Z)
• Learning Extrinsic Dexterity with Parameterized Manipulation Primitives [8.7221770019454]
  We learn a sequence of actions that utilize the environment to change the object's pose. Our approach can control the object's state through exploiting interactions between the object, the gripper, and the environment. We evaluate our approach on picking box-shaped objects of various weight, shape, and friction properties from a constrained table-top workspace.
  arXiv  Detail & Related papers  (2023-10-26T21:28:23Z)
• Canonical mapping as a general-purpose object descriptor for robotic manipulation [0.0]
  We propose using canonical mapping as a near-universal and flexible object descriptor. We demonstrate that common object representations can be derived from a single pre-trained canonical mapping model. We perform a multi-stage experiment using two robot arms that demonstrate the robustness of the perception approach.
  arXiv  Detail & Related papers  (2023-03-02T15:09:25Z)
• TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulation [14.011777717620282]
  We propose a vision-based system that learns to estimate the cross-pose between two objects for a given manipulation task. We demonstrate our method's capability to generalize to unseen objects, in some cases after training on only 10 demonstrations in the real world.
  arXiv  Detail & Related papers  (2022-11-17T04:06:16Z)
• Semantically Grounded Object Matching for Robust Robotic Scene Rearrangement [21.736603698556042]
  We present a novel approach to object matching that uses a large pre-trained vision-language model to match objects in a cross-instance setting. We demonstrate that this provides considerably improved matching performance in cross-instance settings.
  arXiv  Detail & Related papers  (2021-11-15T18:39:43Z)
• V-MAO: Generative Modeling for Multi-Arm Manipulation of Articulated Objects [51.79035249464852]
  We present a framework for learning multi-arm manipulation of articulated objects. Our framework includes a variational generative model that learns contact point distribution over object rigid parts for each robot arm.
  arXiv  Detail & Related papers  (2021-11-07T02:31:09Z)
• Consolidating Kinematic Models to Promote Coordinated Mobile Manipulations [96.03270112422514]
  We construct a Virtual Kinematic Chain (VKC) that consolidates the kinematics of the mobile base, the arm, and the object to be manipulated in mobile manipulations. A mobile manipulation task is represented by altering the state of the constructed VKC, which can be converted to a motion planning problem.
  arXiv  Detail & Related papers  (2021-08-03T02:59:41Z)
• Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation [26.24940293693809]
  We propose using reinforcement learning to compose hierarchical object-centric controllers for manipulation tasks. Experiments in both simulation and real world show how the proposed approach leads to improved sample efficiency, zero-shot generalization, and simulation-to-reality transfer without fine-tuning.
  arXiv  Detail & Related papers  (2020-11-09T18:38:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.