CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context

• URL: http://arxiv.org/abs/2003.11696v2
• Date: Sun, 1 Nov 2020 04:21:48 GMT
• Title: CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context
• Authors: Wenyu Zhang, Skyler Seto, Devesh K. Jha
• Abstract summary: We study the problem of designing deep learning agents which can generalize their models of the physical world by building context-aware models. We present context-aware zero shot learning (CAZSL, pronounced as casual) models, an approach utilizing a Siamese network, embedding space and regularization based on context variables. We test our proposed learning algorithm on the recently released Omnipush datatset that allows testing of meta-learning capabilities.
• Score: 13.217582954907234
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning accurate models of the physical world is required for a lot of robotic manipulation tasks. However, during manipulation, robots are expected to interact with unknown workpieces so that building predictive models which can generalize over a number of these objects is highly desirable. In this paper, we study the problem of designing deep learning agents which can generalize their models of the physical world by building context-aware learning models. The purpose of these agents is to quickly adapt and/or generalize their notion of physics of interaction in the real world based on certain features about the interacting objects that provide different contexts to the predictive models. With this motivation, we present context-aware zero shot learning (CAZSL, pronounced as casual) models, an approach utilizing a Siamese network architecture, embedding space masking and regularization based on context variables which allows us to learn a model that can generalize to different parameters or features of the interacting objects. We test our proposed learning algorithm on the recently released Omnipush datatset that allows testing of meta-learning capabilities using low-dimensional data. Codes for CAZSL are available at https://www.merl.com/research/license/CAZSL.

Related papers

• SOLD: Reinforcement Learning with Slot Object-Centric Latent Dynamics [16.020835290802548]
  Slot-Attention for Object-centric Latent Dynamics is a novel algorithm that learns object-centric dynamics models from pixel inputs. We demonstrate that the structured latent space not only improves model interpretability but also provides a valuable input space for behavior models to reason over. Our results show that SOLD outperforms DreamerV3, a state-of-the-art model-based RL algorithm, across a range of benchmark robotic environments.
  arXiv  Detail & Related papers  (2024-10-11T14:03:31Z)
• Slot Structured World Models [0.0]
  State-of-the-art approaches use a feedforward encoder to extract object embeddings and a latent graph neural network to model the interaction between these object embeddings. We introduce it Slot Structured World Models (SSWM), a class of world models that combines an it object-centric encoder with a latent graph-based dynamics model.
  arXiv  Detail & Related papers  (2024-01-08T21:19:30Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models. Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning. Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Curriculum-Based Imitation of Versatile Skills [15.97723808124603]
  Learning skills by imitation is a promising concept for the intuitive teaching of robots. A common way to learn such skills is to learn a parametric model by maximizing the likelihood given the demonstrations. Yet, human demonstrations are often multi-modal, i.e., the same task is solved in multiple ways.
  arXiv  Detail & Related papers  (2023-04-11T12:10:41Z)
• Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents [111.15288256221764]
  Grounded-decoding project aims to solve complex, long-horizon tasks in a robotic setting by leveraging the knowledge of both models. We frame this as a problem similar to probabilistic filtering: decode a sequence that both has high probability under the language model and high probability under a set of grounded model objectives. We demonstrate how such grounded models can be obtained across three simulation and real-world domains, and that the proposed decoding strategy is able to solve complex, long-horizon tasks in a robotic setting by leveraging the knowledge of both models.
  arXiv  Detail & Related papers  (2023-03-01T22:58:50Z)
• RT-1: Robotics Transformer for Real-World Control at Scale [98.09428483862165]
  We present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks.
  arXiv  Detail & Related papers  (2022-12-13T18:55:15Z)
• Learning Multi-Object Dynamics with Compositional Neural Radiance Fields [63.424469458529906]
  We present a method to learn compositional predictive models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural networks. NeRFs have become a popular choice for representing scenes due to their strong 3D prior. For planning, we utilize RRTs in the learned latent space, where we can exploit our model and the implicit object encoder to make sampling the latent space informative and more efficient.
  arXiv  Detail & Related papers  (2022-02-24T01:31:29Z)
• Factored World Models for Zero-Shot Generalization in Robotic Manipulation [7.258229016768018]
  We learn to generalize over robotic pick-and-place tasks using object-factored world models. We use a residual stack of graph neural networks that receive action information at multiple levels in both their node and edge neural networks. We show that an ensemble of our models can be used to plan for tasks involving up to 12 pick and place actions using search.
  arXiv  Detail & Related papers  (2022-02-10T21:26:11Z)
• Model-Based Visual Planning with Self-Supervised Functional Distances [104.83979811803466]
  We present a self-supervised method for model-based visual goal reaching. Our approach learns entirely using offline, unlabeled data. We find that this approach substantially outperforms both model-free and model-based prior methods.
  arXiv  Detail & Related papers  (2020-12-30T23:59:09Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.