Amortized Low-Rank Adaptation for Model-Based Reinforcement Learning

• URL: http://arxiv.org/abs/2609.12278v1
• Date: Thu, 10 Sep 2026 23:16:40 GMT
• Title: Amortized Low-Rank Adaptation for Model-Based Reinforcement Learning
• Abstract summary: World models let agents plan by predicting the consequences of their actions, but changes in the environment can make them inaccurate.<n>Existing approaches trade off computational cost against expressivity, i.e., the range of models a method can produce.<n>We present CLAW, which addresses this tradeoff by using a hypernetwork to generate adapters at test time.
• Score: 49.3087425299849
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World models let agents plan by predicting the consequences of their actions, but changes in the environment can make them inaccurate. We study the problem of adapting a world model to an unknown test-time environment, drawn from a known environment family, using only a few episodes of interaction. Existing approaches trade off computational cost against expressivity, i.e., the range of models a method can produce. For example, in-context learning is computationally cheap but limited in expressivity, and gradient-based adaptation is expressive but computationally expensive. We present CLAW (Context-conditioned Low-rank Adaptation of World models), which addresses this tradeoff by using a hypernetwork to generate low-rank (LoRA) adapters at test time. During pretraining, we simulate adaptation to a variety of environments and jointly train the hypernetwork and base world model. At test time, we freeze the base model and use a forward pass of the hypernetwork to generate adapters from a small batch of test-time transitions. We evaluate CLAW in locomotion and manipulation environment families that vary in dynamics, embodiment, and reward. We show that, using only seconds of test-time data, CLAW outperforms gradient-based adaptation and in-context learning during online adaptation. We also show that CLAW avoids overfitting in data-scarce regimes, that its advantage comes from the expressive adapters rather than context conditioning, and that pretraining the hypernetwork jointly with the base model outperforms training it post hoc.

Related papers

• The Golden Subspace: Where Efficiency Meets Generalization in Continual Test-Time Adaptation [86.83038620308423]
  Continual Test-Time Adaptation (CTTA) aims to enable models to adapt online to unlabeled data streams under distribution shift without accessing source data.<n>We propose Guided Online Low-rank Directional adaptation (GOLD), which uses a lightweight adapter to project features onto the golden subspace and learns a compact scaling vector while the subspace is dynamically updated via AGOP.
  arXiv  Detail & Related papers  (2026-03-23T12:48:38Z)
• ChameleonLLM: Batch-Aware Dynamic Low-Rank Adaptation via Inference-Time Clusters [3.729242965449096]
  This paper introduces ChameleonLLM, a novel framework that enables inference-time adaptation of large language models.<n>Unlike traditional fine-tuning approaches such as Low-Rank Adaptation (LoRA), our method dynamically generates adaptive modifications to the decoder weights.<n>By intelligently grouping similar inputs and computing context-aware low-rank updates via a hyper-network, ChameleonLLM achieves significant performance gains.
  arXiv  Detail & Related papers  (2025-02-06T18:57:06Z)
• Adaptive Adapter Routing for Long-Tailed Class-Incremental Learning [55.384428765798496]
  New data exhibits a long-tailed distribution, such as e-commerce platform reviews. This necessitates continuous model learning imbalanced data without forgetting. We introduce AdaPtive Adapter RouTing (APART) as an exemplar-free solution for LTCIL.
  arXiv  Detail & Related papers  (2024-09-11T17:52:00Z)
• CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning [17.614980614656407]
  We propose Continual Generative training for Incremental prompt-Learning. We exploit Variational Autoencoders to learn class-conditioned distributions. We show that such a generative replay approach can adapt to new tasks while improving zero-shot capabilities.
  arXiv  Detail & Related papers  (2024-07-22T16:51:28Z)
• Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion [10.322832012497722]
  Class-incremental learning is a challenging problem, where the goal is to train a model that can classify data from an increasing number of classes over time. With the advancement of vision-language pre-trained models such as CLIP, they demonstrate good generalization ability. However, further adaptation to downstream tasks by simply fine-tuning the model leads to severe forgetting. Most existing works with pre-trained models assume that the forgetting of old classes is uniform when the model acquires new knowledge.
  arXiv  Detail & Related papers  (2024-07-19T09:20:33Z)
• Efficient Open Set Single Image Test Time Adaptation of Vision Language Models [15.621092104244003]
  Adapting models to dynamic, real-world environments is a critical challenge in deep learning.<n>We propose ROSITA, a novel framework that leverages dynamically updated feature banks to identify reliable test samples.<n>Our approach effectively adapts models to domain shifts for known classes while rejecting unfamiliar samples.
  arXiv  Detail & Related papers  (2024-06-01T16:21:42Z)
• Test-Time Training for Semantic Segmentation with Output Contrastive Loss [12.535720010867538]
  Deep learning-based segmentation models have achieved impressive performance on public benchmarks, but generalizing well to unseen environments remains a major challenge. This paper introduces Contrastive Loss (OCL), known for its capability to learn robust and generalized representations, to stabilize the adaptation process. Our method excels even when applied to models initially pre-trained using domain adaptation methods on test domain data, showcasing its resilience and adaptability.
  arXiv  Detail & Related papers  (2023-11-14T03:13:47Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment [55.91291559442884]
  This work develops a methodology that enables data-driven methods to continuously learn and optimize in a dynamic environment. We propose to build the notion of continual learning into the modeling process of learning wireless systems. Our design is based on a novel min-max formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2020-11-16T08:24:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.