Collective Intelligence for 2D Push Manipulations with Mobile Robots

• URL: http://arxiv.org/abs/2211.15136v3
• Date: Wed, 5 Apr 2023 03:16:26 GMT
• Title: Collective Intelligence for 2D Push Manipulations with Mobile Robots
• Authors: So Kuroki, Tatsuya Matsushima, Jumpei Arima, Hiroki Furuta, Yutaka Matsuo, Shixiang Shane Gu, Yujin Tang
• Abstract summary: We show that by distilling a planner from a differentiable soft-body physics simulator into an attention-based neural network, our multi-robot push manipulation system achieves better performance than baselines. Our system also generalizes to configurations not seen during training and is able to adapt toward task completions when external turbulence and environmental changes are applied.
• Score: 18.937030864563752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While natural systems often present collective intelligence that allows them to self-organize and adapt to changes, the equivalent is missing in most artificial systems. We explore the possibility of such a system in the context of cooperative 2D push manipulations using mobile robots. Although conventional works demonstrate potential solutions for the problem in restricted settings, they have computational and learning difficulties. More importantly, these systems do not possess the ability to adapt when facing environmental changes. In this work, we show that by distilling a planner derived from a differentiable soft-body physics simulator into an attention-based neural network, our multi-robot push manipulation system achieves better performance than baselines. In addition, our system also generalizes to configurations not seen during training and is able to adapt toward task completions when external turbulence and environmental changes are applied. Supplementary videos can be found on our project website: https://sites.google.com/view/ciom/home

Related papers

• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• SPIN: Simultaneous Perception, Interaction and Navigation [33.408010508592824]
  We present a reactive mobile manipulation framework that uses an active visual system to consciously perceive and react to its environment. Similar to how humans leverage whole-body and hand-eye coordination, we develop a mobile manipulator that exploits its ability to move and see.
  arXiv  Detail & Related papers  (2024-05-13T17:59:36Z)
• LPAC: Learnable Perception-Action-Communication Loops with Applications to Coverage Control [80.86089324742024]
  We propose a learnable Perception-Action-Communication (LPAC) architecture for the problem. CNN processes localized perception; a graph neural network (GNN) facilitates robot communications. Evaluations show that the LPAC models outperform standard decentralized and centralized coverage control algorithms.
  arXiv  Detail & Related papers  (2024-01-10T00:08:00Z)
• Harmonic Mobile Manipulation [35.82197562695662]
  HarmonicMM is an end-to-end learning method that optimize both navigation and manipulation. Our contributions include a new benchmark for mobile manipulation and the successful deployment with only RGB visual observation.
  arXiv  Detail & Related papers  (2023-12-11T18:54:42Z)
• Bio-inspired spike-based Hippocampus and Posterior Parietal Cortex models for robot navigation and environment pseudo-mapping [52.77024349608834]
  This work proposes a spike-based robotic navigation and environment pseudomapping system. The hippocampus is in charge of maintaining a representation of an environment state map, and the PPC is in charge of local decision-making. This is the first implementation of an environment pseudo-mapping system with dynamic learning based on a bio-inspired hippocampal memory.
  arXiv  Detail & Related papers  (2023-05-22T10:20:34Z)
• ExAug: Robot-Conditioned Navigation Policies via Geometric Experience Augmentation [73.63212031963843]
  We propose a novel framework, ExAug, to augment the experiences of different robot platforms from multiple datasets in diverse environments. The trained policy is evaluated on two new robot platforms with three different cameras in indoor and outdoor environments with obstacles.
  arXiv  Detail & Related papers  (2022-10-14T01:32:15Z)
• Autonomous Intraluminal Navigation of a Soft Robot using Deep-Learning-based Visual Servoing [13.268863900187025]
  We present a synergic solution for intraluminal navigation consisting of a 3D printed endoscopic soft robot. Visual servoing, based on Convolutional Neural Networks (CNNs), is used to achieve the autonomous navigation task. The proposed robot is validated in anatomical phantoms in different path configurations.
  arXiv  Detail & Related papers  (2022-07-01T13:17:45Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• COCOI: Contact-aware Online Context Inference for Generalizable Non-planar Pushing [87.7257446869134]
  General contact-rich manipulation problems are long-standing challenges in robotics. Deep reinforcement learning has shown great potential in solving robot manipulation tasks. We propose COCOI, a deep RL method that encodes a context embedding of dynamics properties online.
  arXiv  Detail & Related papers  (2020-11-23T08:20:21Z)
• Deep Reactive Planning in Dynamic Environments [20.319894237644558]
  A robot can learn an end-to-end policy which can adapt to changes in the environment during execution. We present a method that can achieve such behavior by combining traditional kinematic planning, deep learning, and deep reinforcement learning. We demonstrate the proposed approach for several reaching and pick-and-place tasks in simulation, as well as on a real system of a 6-DoF industrial manipulator.
  arXiv  Detail & Related papers  (2020-10-31T00:46:13Z)
• How to reduce computation time while sparing performance during robot navigation? A neuro-inspired architecture for autonomous shifting between model-based and model-free learning [1.3854111346209868]
  We present a novel arbitration mechanism between learning systems that explicitly measures performance and cost. We find that the robot can adapt to environment changes by switching between learning systems so as to maintain a high performance. When the task is stable, the robot also autonomously shifts to the least costly system, which leads to a drastic reduction in computation cost while keeping a high performance.
  arXiv  Detail & Related papers  (2020-04-30T11:29:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.