Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads

• URL: http://arxiv.org/abs/2004.11345v2
• Date: Tue, 2 Feb 2021 06:32:03 GMT
• Title: Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads
• Authors: Suneel Belkhale, Rachel Li, Gregory Kahn, Rowan McAllister, Roberto Calandra, Sergey Levine
• Abstract summary: Transporting suspended payloads is challenging for autonomous aerial vehicles. We propose a meta-learning approach that "learns how to learn" models of altered dynamics within seconds of post-connection flight data.
• Score: 69.21503033239985
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transporting suspended payloads is challenging for autonomous aerial vehicles because the payload can cause significant and unpredictable changes to the robot's dynamics. These changes can lead to suboptimal flight performance or even catastrophic failure. Although adaptive control and learning-based methods can in principle adapt to changes in these hybrid robot-payload systems, rapid mid-flight adaptation to payloads that have a priori unknown physical properties remains an open problem. We propose a meta-learning approach that "learns how to learn" models of altered dynamics within seconds of post-connection flight data. Our experiments demonstrate that our online adaptation approach outperforms non-adaptive methods on a series of challenging suspended payload transportation tasks. Videos and other supplemental material are available on our website: https://sites.google.com/view/meta-rl-for-flight

Related papers

• MULE: Multi-terrain and Unknown Load Adaptation for Effective Quadrupedal Locomotion [1.479858319622657]
  Quadrupedal robots are increasingly deployed for load-carrying tasks across diverse terrains. We propose an Adaptive Reinforcement Learning framework that enables quadrupedal robots to adapt to both varying payloads and diverse terrains. We validate the proposed approach through large-scale simulation experiments in Isaac Gym and real-world hardware deployment on a Unitree Go1 quadruped.
  arXiv  Detail & Related papers  (2025-05-01T12:41:35Z)
• Action Flow Matching for Continual Robot Learning [57.698553219660376]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks. We introduce a generative framework leveraging flow matching for online robot dynamics model alignment. We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)
• Meta-Learning Online Dynamics Model Adaptation in Off-Road Autonomous Driving [9.515695438588855]
  High-speed off-road autonomous driving presents unique challenges due to complex, evolving terrain characteristics. We propose a novel framework that combines a Kalman filter-based online adaptation scheme with meta-learned parameters to address these challenges. Our results underscore the effectiveness of meta-learned dynamics model adaptation, advancing the development of reliable autonomous systems.
  arXiv  Detail & Related papers  (2025-04-23T17:51:36Z)
• First, Learn What You Don't Know: Active Information Gathering for Driving at the Limits of Handling [38.468291768795865]
  In unstable systems, online adaptation may not be fast enough to ensure reliable simultaneous learning and control. We present a Bayesian meta-learning MPC framework to enable rapid online adaptation. Experiments on a Toyota Supra show that the framework enables reliable control in dynamic drifting maneuvers.
  arXiv  Detail & Related papers  (2024-10-31T18:02:30Z)
• Physics Enhanced Residual Policy Learning (PERPL) for safety cruising in mixed traffic platooning under actuator and communication delay [8.172286651098027]
  Linear control models have gained extensive application in vehicle control due to their simplicity, ease of use, and support for stability analysis. Reinforcement learning (RL) models, on the other hand, offer adaptability but suffer from a lack of interpretability and generalization capabilities. This paper aims to develop a family of RL-based controllers enhanced by physics-informed policies.
  arXiv  Detail & Related papers  (2024-09-23T23:02:34Z)
• Model-Based Reinforcement Learning with Multi-Task Offline Pretraining [59.82457030180094]
  We present a model-based RL method that learns to transfer potentially useful dynamics and action demonstrations from offline data to a novel task. The main idea is to use the world models not only as simulators for behavior learning but also as tools to measure the task relevance. We demonstrate the advantages of our approach compared with the state-of-the-art methods in Meta-World and DeepMind Control Suite.
  arXiv  Detail & Related papers  (2023-06-06T02:24:41Z)
• OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation [50.59541802645156]
  Operational Space Control (OSC) has been used as an effective task-space controller for manipulation. We propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of OSC that compensates for modeling errors. We evaluate our method on a variety of simulated manipulation problems, and find substantial improvements over an array of controller baselines.
  arXiv  Detail & Related papers  (2021-10-02T01:21:38Z)
• Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms [60.59764170868101]
  Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
  arXiv  Detail & Related papers  (2021-03-05T14:16:20Z)
• Meta-Learning-Based Robust Adaptive Flight Control Under Uncertain Wind Conditions [13.00214468719929]
  Realtime model learning is challenging for complex dynamical systems, such as drones flying in variable wind conditions. We propose an online composite adaptation method that treats outputs from a deep neural network as a set of basis functions. We validate our approach by flying a drone in an open air wind tunnel under varying wind conditions and along challenging trajectories.
  arXiv  Detail & Related papers  (2021-03-02T18:43:59Z)
• Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments [3.5309638744466167]
  This work developed a meta-learning approach that adapts the control policy on the fly to different changing conditions for robust locomotion. The proposed method constantly updates the interaction model, samples feasible sequences of actions of estimated the state-action trajectories, and then applies the optimal actions to maximize the reward.
  arXiv  Detail & Related papers  (2021-01-19T12:57:12Z)
• Never Stop Learning: The Effectiveness of Fine-Tuning in Robotic Reinforcement Learning [109.77163932886413]
  We show how to adapt vision-based robotic manipulation policies to new variations by fine-tuning via off-policy reinforcement learning. This adaptation uses less than 0.2% of the data necessary to learn the task from scratch. We find that our approach of adapting pre-trained policies leads to substantial performance gains over the course of fine-tuning.
  arXiv  Detail & Related papers  (2020-04-21T17:57:04Z)
• Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning [65.88200578485316]
  We present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. Our method significantly improves adaptation to changes in dynamics in high noise settings. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics.
  arXiv  Detail & Related papers  (2020-03-02T22:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.