Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters

• URL: http://arxiv.org/abs/2503.18684v2
• Date: Thu, 27 Mar 2025 13:19:46 GMT
• Title: Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters
• Authors: Ruiqi Zhu, Endong Sun, Guanhe Huang, Oya Celiktutan,
• Abstract summary: Continual adaptation is essential for general autonomous agents.<n>Online Meta-Learned adapters (OMLA) facilitate knowledge transfer from previously learned tasks to current learning tasks.
• Score: 0.24106250158920473
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual adaptation is essential for general autonomous agents. For example, a household robot pretrained with a repertoire of skills must still adapt to unseen tasks specific to each household. Motivated by this, building upon parameter-efficient fine-tuning in language models, prior works have explored lightweight adapters to adapt pretrained policies, which can preserve learned features from the pretraining phase and demonstrate good adaptation performances. However, these approaches treat task learning separately, limiting knowledge transfer between tasks. In this paper, we propose Online Meta-Learned adapters (OMLA). Instead of applying adapters directly, OMLA can facilitate knowledge transfer from previously learned tasks to current learning tasks through a novel meta-learning objective. Extensive experiments in both simulated and real-world environments demonstrate that OMLA can lead to better adaptation performances compared to the baseline methods. The project link: https://ricky-zhu.github.io/OMLA/.

Related papers

• Linked Adapters: Linking Past and Future to Present for Effective Continual Learning [3.7166121807265045]
  Continual learning allows the system to learn and adapt to new tasks while retaining the knowledge acquired from previous tasks.<n>Deep learning models suffer from catastrophic forgetting of knowledge learned from earlier tasks while learning a new task.<n>We propose a novel approach that allows knowledge transfer through a weighted attention mechanism to other task-specific adapters.
  arXiv  Detail & Related papers  (2024-12-14T05:25:17Z)
• From Instance Training to Instruction Learning: Task Adapters Generation from Instructions [29.452006810725184]
  This paper focuses on simulating human learning to address the shortcomings of instance training. We introduce Task Adapters Generation from Instructions (TAGI), which automatically constructs the task-specific model. We evaluate TAGI on the Super-Natural Instructions and P3 datasets.
  arXiv  Detail & Related papers  (2024-06-18T08:14:28Z)
• Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning [21.19820308728003]
  Continual learning (CL) aims to continually accumulate knowledge from a non-stationary data stream without catastrophic forgetting of learned knowledge. Current PTM-based CL methods perform effective continual adaptation on downstream tasks by adding learnable adapters or prompts upon the frozen PTMs. We propose Self-Expansion of pre-trained models with Modularized Adaptation (SEMA), a novel approach to enhance the control of stability-plasticity balance in PTM-based CL.
  arXiv  Detail & Related papers  (2024-03-27T17:59:21Z)
• Class Incremental Learning with Pre-trained Vision-Language Models [59.15538370859431]
  We propose an approach to exploiting pre-trained vision-language models (e.g. CLIP) that enables further adaptation. Experiments on several conventional benchmarks consistently show a significant margin of improvement over the current state-of-the-art.
  arXiv  Detail & Related papers  (2023-10-31T10:45:03Z)
• Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving [103.745551954983]
  In this paper, we investigate the transfer performance of various types of self-supervised methods, including MoCo and SimCLR, on three downstream tasks. We find that their performances are sub-optimal or even lag far behind the single-task baseline. We propose a simple yet effective pretrain-adapt-finetune paradigm for general multi-task training.
  arXiv  Detail & Related papers  (2022-09-19T12:15:31Z)
• KALA: Knowledge-Augmented Language Model Adaptation [65.92457495576141]
  We propose a novel domain adaption framework for pre-trained language models (PLMs) Knowledge-Augmented Language model Adaptation (KALA) modulates the intermediate hidden representations of PLMs with domain knowledge. Results show that, despite being computationally efficient, our KALA largely outperforms adaptive pre-training.
  arXiv  Detail & Related papers  (2022-04-22T08:11:59Z)
• Fully Online Meta-Learning Without Task Boundaries [80.09124768759564]
  We study how meta-learning can be applied to tackle online problems of this nature. We propose a Fully Online Meta-Learning (FOML) algorithm, which does not require any ground truth knowledge about the task boundaries. Our experiments show that FOML was able to learn new tasks faster than the state-of-the-art online learning methods.
  arXiv  Detail & Related papers  (2022-02-01T07:51:24Z)
• AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters [34.86139827292556]
  Open-access dissemination of pretrained language models has led to a democratization of state-of-the-art natural language processing (NLP) research. This also allows people outside of NLP to use such models and adapt them to specific use-cases. In this work, we aim to overcome this gap by providing a tool which allows researchers to leverage pretrained models without writing a single line of code.
  arXiv  Detail & Related papers  (2021-08-18T11:56:01Z)
• Linear Representation Meta-Reinforcement Learning for Instant Adaptation [20.711877803169134]
  This paper introduces Fast Linearized Adaptive Policy (FLAP) FLAP is a new meta-reinforcement learning (meta-RL) method that is able to extrapolate well to out-of-distribution tasks.
  arXiv  Detail & Related papers  (2021-01-12T20:56:34Z)
• Meta-learning the Learning Trends Shared Across Tasks [123.10294801296926]
  Gradient-based meta-learning algorithms excel at quick adaptation to new tasks with limited data. Existing meta-learning approaches only depend on the current task information during the adaptation. We propose a 'Path-aware' model-agnostic meta-learning approach.
  arXiv  Detail & Related papers  (2020-10-19T08:06:47Z)
• Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning [74.07455280246212]
  Continual learning studies agents that learn from streams of tasks without forgetting previous ones while adapting to new ones. We show that current continual learning, meta-learning, meta-continual learning, and continual-meta learning techniques fail in this new scenario. We propose Continual-MAML, an online extension of the popular MAML algorithm as a strong baseline for this scenario.
  arXiv  Detail & Related papers  (2020-03-12T15:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.