Towards Multi-Morphology Controllers with Diversity and Knowledge Distillation

• URL: http://arxiv.org/abs/2404.14625v1
• Date: Mon, 22 Apr 2024 23:40:03 GMT
• Title: Towards Multi-Morphology Controllers with Diversity and Knowledge Distillation
• Authors: Alican Mertan, Nick Cheney,
• Abstract summary: We present a pipeline that distills many single-task/single-morphology teacher controllers into a single multi-morphology controller. The distilled controller scales well with the number of teachers/morphologies and shows emergent properties. It generalizes to unseen morphologies in a zero-shot manner, providing robustness to morphological perturbations and instant damage recovery.
• Score: 0.24554686192257422
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Finding controllers that perform well across multiple morphologies is an important milestone for large-scale robotics, in line with recent advances via foundation models in other areas of machine learning. However, the challenges of learning a single controller to control multiple morphologies make the `one robot one task' paradigm dominant in the field. To alleviate these challenges, we present a pipeline that: (1) leverages Quality Diversity algorithms like MAP-Elites to create a dataset of many single-task/single-morphology teacher controllers, then (2) distills those diverse controllers into a single multi-morphology controller that performs well across many different body plans by mimicking the sensory-action patterns of the teacher controllers via supervised learning. The distilled controller scales well with the number of teachers/morphologies and shows emergent properties. It generalizes to unseen morphologies in a zero-shot manner, providing robustness to morphological perturbations and instant damage recovery. Lastly, the distilled controller is also independent of the teacher controllers -- we can distill the teacher's knowledge into any controller model, making our approach synergistic with architectural improvements and existing training algorithms for teacher controllers.

Related papers

• A comparison of controller architectures and learning mechanisms for arbitrary robot morphologies [2.884244918665901]
  What combination of a robot controller and a learning method should be used, if the morphology of the learning robot is not known in advance? We perform an experimental comparison of three controller-and-learner combinations. We compare their efficacy, efficiency, and robustness.
  arXiv  Detail & Related papers  (2023-09-25T07:11:43Z)
• Evolving generalist controllers to handle a wide range of morphological variations [1.4425878137951238]
  The study of robustness and generalizability of artificial neural networks (ANNs) has remained limited. Unexpected morphological or environmental changes during operation can risk failure if the ANN controllers are unable to handle these changes. This paper proposes an algorithm that aims to enhance the robustness and generalizability of the controllers.
  arXiv  Detail & Related papers  (2023-09-18T23:06:19Z)
• Universal Morphology Control via Contextual Modulation [52.742056836818136]
  Learning a universal policy across different robot morphologies can significantly improve learning efficiency and generalization in continuous control. Existing methods utilize graph neural networks or transformers to handle heterogeneous state and action spaces across different morphologies. We propose a hierarchical architecture to better model this dependency via contextual modulation.
  arXiv  Detail & Related papers  (2023-02-22T00:04:12Z)
• Is Disentanglement enough? On Latent Representations for Controllable Music Generation [78.8942067357231]
  In the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes.
  arXiv  Detail & Related papers  (2021-08-01T18:37:43Z)
• Versatile modular neural locomotion control with fast learning [6.85316573653194]
  Legged robots have significant potential to operate in highly unstructured environments. Currently, controllers must be either manually designed for specific robots or automatically designed via machine learning methods. We propose a simple yet versatile modular neural control structure with fast learning.
  arXiv  Detail & Related papers  (2021-07-16T12:12:28Z)
• Morpho-evolution with learning using a controller archive as an inheritance mechanism [4.364091192392204]
  We propose a framework that combines an evolutionary algorithm to generate body-plans and a learning algorithm to optimise the parameters of a neural controller. By inheriting an appropriate controller from an archive rather than learning from a randomly initialised one, we show that both the speed and magnitude of learning increases over time. The framework also provides new insights into the complex interactions between evolution and learning, and the role of morphological intelligence in robot design.
  arXiv  Detail & Related papers  (2021-04-09T09:32:36Z)
• A Meta-Reinforcement Learning Approach to Process Control [3.9146761527401424]
  Meta-learning aims to quickly adapt models, such as neural networks, to perform new tasks. We construct a controller and meta-train the controller using a latent context variable through a separate embedding neural network. In both cases, our meta-learning algorithm adapts very quickly to new tasks, outperforming a regular DRL controller trained from scratch.
  arXiv  Detail & Related papers  (2021-03-25T18:20:56Z)
• Machine Learning for Mechanical Ventilation Control [52.65490904484772]
  We consider the problem of controlling an invasive mechanical ventilator for pressure-controlled ventilation. A PID controller must let air in and out of a sedated patient's lungs according to a trajectory of airway pressures specified by a clinician. We show that our controllers are able to track target pressure waveforms significantly better than PID controllers.
  arXiv  Detail & Related papers  (2021-02-12T21:23:33Z)
• Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion [95.1825179206694]
  We present a framework that synthesizes robust controllers for a quadruped robot. A high-level controller learns to choose from a set of primitives in response to changes in the environment. A low-level controller that utilizes an established control method to robustly execute the primitives.
  arXiv  Detail & Related papers  (2020-09-21T16:49:26Z)
• Data-driven Koopman Operators for Model-based Shared Control of Human-Machine Systems [66.65503164312705]
  We present a data-driven shared control algorithm that can be used to improve a human operator's control of complex machines. Both the dynamics and information about the user's interaction are learned from observation through the use of a Koopman operator. We find that model-based shared control significantly improves task and control metrics when compared to a natural learning, or user only, control paradigm.
  arXiv  Detail & Related papers  (2020-06-12T14:14:07Z)
• Improving Input-Output Linearizing Controllers for Bipedal Robots via Reinforcement Learning [85.13138591433635]
  The main drawbacks of input-output linearizing controllers are the need for precise dynamics models and not being able to account for input constraints. In this paper, we address both challenges for the specific case of bipedal robot control by the use of reinforcement learning techniques.
  arXiv  Detail & Related papers  (2020-04-15T18:15:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.