Object and Relation Centric Representations for Push Effect Prediction

• URL: http://arxiv.org/abs/2102.02100v1
• Date: Wed, 3 Feb 2021 15:09:12 GMT
• Title: Object and Relation Centric Representations for Push Effect Prediction
• Authors: Ahmet E. Tekden, Aykut Erdem, Erkut Erdem, Tamim Asfour, Emre Ugur
• Abstract summary: Pushing is an essential non-prehensile manipulation skill used for tasks ranging from pre-grasp manipulation to scene rearrangement. We propose a graph neural network based framework for effect prediction and parameter estimation of pushing actions. Our framework is validated both in real and simulated environments containing different shaped multi-part objects connected via different types of joints and objects with different masses.
• Score: 18.990827725752496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pushing is an essential non-prehensile manipulation skill used for tasks ranging from pre-grasp manipulation to scene rearrangement, reasoning about object relations in the scene, and thus pushing actions have been widely studied in robotics. The effective use of pushing actions often requires an understanding of the dynamics of the manipulated objects and adaptation to the discrepancies between prediction and reality. For this reason, effect prediction and parameter estimation with pushing actions have been heavily investigated in the literature. However, current approaches are limited because they either model systems with a fixed number of objects or use image-based representations whose outputs are not very interpretable and quickly accumulate errors. In this paper, we propose a graph neural network based framework for effect prediction and parameter estimation of pushing actions by modeling object relations based on contacts or articulations. Our framework is validated both in real and simulated environments containing different shaped multi-part objects connected via different types of joints and objects with different masses. Our approach enables the robot to predict and adapt the effect of a pushing action as it observes the scene. Further, we demonstrate 6D effect prediction in the lever-up action in the context of robot-based hard-disk disassembly.

Related papers

• Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking [59.87033229815062]
  Articulated object manipulation requires precise object interaction, where the object's axis must be carefully considered. Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics. We present a closed-loop pipeline integrating interactive perception with online axis estimation from segmented 3D point clouds.
  arXiv  Detail & Related papers  (2024-09-24T17:59:56Z)
• PEAR: Phrase-Based Hand-Object Interaction Anticipation [20.53329698350243]
  First-person hand-object interaction anticipation aims to predict the interaction process based on current scenes and prompts. Existing research typically anticipates only interaction intention while neglecting manipulation. We propose a novel model, PEAR, which jointly anticipates interaction intention and manipulation.
  arXiv  Detail & Related papers  (2024-07-31T10:28:49Z)
• RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing [38.97168020979433]
  We introduce an approach that combines visual and tactile sensing for robotic manipulation by learning a neural, tactile-informed dynamics model. Our proposed framework, RoboPack, employs a recurrent graph neural network to estimate object states. We demonstrate our approach on a real robot equipped with a compliant Soft-Bubble tactile sensor on non-prehensile manipulation and dense packing tasks.
  arXiv  Detail & Related papers  (2024-07-01T16:08:37Z)
• Learning Manipulation by Predicting Interaction [85.57297574510507]
  We propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction. The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms.
  arXiv  Detail & Related papers  (2024-06-01T13:28:31Z)
• Event-based Vision for Early Prediction of Manipulation Actions [0.7699714865575189]
  Neuromorphic visual sensors are artificial retinas that sequences output of events when brightness changes occur in the scene. In this study, we introduce an event-based dataset on fine-grained manipulation actions. We also perform an experimental study on the use of transformers for action prediction with events.
  arXiv  Detail & Related papers  (2023-07-26T17:50:17Z)
• H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactions [62.510951695174604]
  "Hypothesize, Simulate, Act, Update, and Repeat" (H-SAUR) is a probabilistic generative framework that generates hypotheses about how objects articulate given input observations. We show that the proposed model significantly outperforms the current state-of-the-art articulated object manipulation framework. We further improve the test-time efficiency of H-SAUR by integrating a learned prior from learning-based vision models.
  arXiv  Detail & Related papers  (2022-10-22T18:39:33Z)
• Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions [82.90906153293585]
  We propose a graph neural network, HO-GCN, to fuse motion data and dynamic descriptors for the prediction task. We show the proposed network that consumes dynamic descriptors can achieve state-of-the-art prediction results and help the network better generalize to unseen objects.
  arXiv  Detail & Related papers  (2022-06-25T09:55:39Z)
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction [63.62263239934777]
  We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction. Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
  arXiv  Detail & Related papers  (2021-12-30T10:45:22Z)
• Learning Transferable Push Manipulation Skills in Novel Contexts [3.1981440103815717]
  We learn a parametric internal model for push interactions that enables a robot to predict the outcome of a physical interaction even in novel contexts. We train on 2 objects for a total of 24,000 pushes in various conditions, and test on 6 objects for a total of 14,400 predicted push outcomes. Our results show that both biased and unbiased predictors can reliably produce predictions in line with the outcomes of a carefully tuned physics simulator.
  arXiv  Detail & Related papers  (2020-07-29T11:48:56Z)
• Causal Discovery in Physical Systems from Videos [123.79211190669821]
  Causal discovery is at the core of human cognition. We consider the task of causal discovery from videos in an end-to-end fashion without supervision on the ground-truth graph structure.
  arXiv  Detail & Related papers  (2020-07-01T17:29:57Z)
• Human and Machine Action Prediction Independent of Object Information [1.0806206850043696]
  We study the role of inter-object relations that change during an action. We predict actions in, on average, less than 64% of the action's duration.
  arXiv  Detail & Related papers  (2020-04-22T12:13:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.