Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment

• URL: http://arxiv.org/abs/2311.01059v1
• Date: Thu, 2 Nov 2023 08:22:28 GMT
• Title: Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment
• Authors: Annie S. Chen, Govind Chada, Laura Smith, Archit Sharma, Zipeng Fu, Sergey Levine, Chelsea Finn
• Abstract summary: We study the problem of adapting on-the-fly to novel scenarios during deployment. Our approach, RObust Autonomous Modulation (ROAM), introduces a mechanism based on the perceived value of pre-trained behaviors. We demonstrate that ROAM enables a robot to adapt rapidly to changes in dynamics both in simulation and on a real Go1 quadruped.
• Score: 92.48012013825988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To succeed in the real world, robots must cope with situations that differ from those seen during training. We study the problem of adapting on-the-fly to such novel scenarios during deployment, by drawing upon a diverse repertoire of previously learned behaviors. Our approach, RObust Autonomous Modulation (ROAM), introduces a mechanism based on the perceived value of pre-trained behaviors to select and adapt pre-trained behaviors to the situation at hand. Crucially, this adaptation process all happens within a single episode at test time, without any human supervision. We provide theoretical analysis of our selection mechanism and demonstrate that ROAM enables a robot to adapt rapidly to changes in dynamics both in simulation and on a real Go1 quadruped, even successfully moving forward with roller skates on its feet. Our approach adapts over 2x as efficiently compared to existing methods when facing a variety of out-of-distribution situations during deployment by effectively choosing and adapting relevant behaviors on-the-fly.

Related papers

• Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models [81.55156507635286]
  Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. Current learning methods often struggle with generalization to the long tail of unexpected situations without heavy human supervision. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection.
  arXiv  Detail & Related papers  (2024-07-02T21:00:30Z)
• ROSO: Improving Robotic Policy Inference via Synthetic Observations [0.0]
  We propose the use of generative artificial intelligence to improve zero-shot performance of a pre-trained policy. Our experiments show that incorporating generative AI into robotic inference significantly improves successful outcomes.
  arXiv  Detail & Related papers  (2023-11-28T10:52:35Z)
• Adaptive Asynchronous Control Using Meta-learned Neural Ordinary Differential Equations [0.0]
  Real-world robotics systems often present challenges that limit the applicability of model-based Reinforcement Learning and Control. We propose a general framework that overcomes those difficulties by meta-learning adaptive dynamics models for continuous-time prediction and control. We present evaluations in two different robot simulations and on a real industrial robot.
  arXiv  Detail & Related papers  (2022-07-25T11:07:29Z)
• Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models [67.78935378952146]
  GenRL is a framework for solving sequential decision-making problems. It exploits the combination of reinforcement learning and latent variable generative models. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training.
  arXiv  Detail & Related papers  (2022-04-18T22:02:32Z)
• Adversarial Training is Not Ready for Robot Learning [55.493354071227174]
  Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations. We show theoretically and experimentally that neural controllers obtained via adversarial training are subjected to three types of defects. Our results suggest that adversarial training is not yet ready for robot learning.
  arXiv  Detail & Related papers  (2021-03-15T07:51:31Z)
• Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments [3.5309638744466167]
  This work developed a meta-learning approach that adapts the control policy on the fly to different changing conditions for robust locomotion. The proposed method constantly updates the interaction model, samples feasible sequences of actions of estimated the state-action trajectories, and then applies the optimal actions to maximize the reward.
  arXiv  Detail & Related papers  (2021-01-19T12:57:12Z)
• Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning [109.77163932886413]
  We show how to adapt vision-based robotic manipulation policies to new variations by fine-tuning via off-policy reinforcement learning. This adaptation uses less than 0.2% of the data necessary to learn the task from scratch. We find that our approach of adapting pre-trained policies leads to substantial performance gains over the course of fine-tuning.
  arXiv  Detail & Related papers  (2020-04-21T17:57:04Z)
• Environmental Adaptation of Robot Morphology and Control through Real-world Evolution [5.08706161686979]
  We apply evolutionary search to yield combinations of morphology and control for our mechanically self-reconfiguring quadruped robot. We evolve solutions on two distinct physical surfaces and analyze the results in terms of both control and morphology.
  arXiv  Detail & Related papers  (2020-03-30T07:57:19Z)
• Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning [65.88200578485316]
  We present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. Our method significantly improves adaptation to changes in dynamics in high noise settings. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics.
  arXiv  Detail & Related papers  (2020-03-02T22:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.