Context Representation via Action-Free Transformer encoder-decoder for Meta Reinforcement Learning

• URL: http://arxiv.org/abs/2512.14057v2
• Date: Wed, 17 Dec 2025 07:40:56 GMT
• Title: Context Representation via Action-Free Transformer encoder-decoder for Meta Reinforcement Learning
• Abstract summary: This paper introduces Context Representation via Action Free Transformer encoder decoder (CRAFT)<n>CRAFT infers task representations solely from sequences of states and rewards.<n>Built on a transformer encoder decoder with rotary positional embeddings, the model captures long range temporal dependencies and robustly encodes both parametric and non-adaptive task variations.
• Score: 6.927922840261954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) enables robots to operate in uncertain environments, but standard approaches often struggle with poor generalization to unseen tasks. Context-adaptive meta reinforcement learning addresses these limitations by conditioning on the task representation, yet they mostly rely on complete action information in the experience making task inference tightly coupled to a specific policy. This paper introduces Context Representation via Action Free Transformer encoder decoder (CRAFT), a belief model that infers task representations solely from sequences of states and rewards. By removing the dependence on actions, CRAFT decouples task inference from policy optimization, supports modular training, and leverages amortized variational inference for scalable belief updates. Built on a transformer encoder decoder with rotary positional embeddings, the model captures long range temporal dependencies and robustly encodes both parametric and non-parametric task variations. Experiments on the MetaWorld ML-10 robotic manipulation benchmark show that CRAFT achieves faster adaptation, improved generalization, and more effective exploration compared to context adaptive meta--RL baselines. These findings highlight the potential of action-free inference as a foundation for scalable RL in robotic control.

Related papers

• MoE-ACT: Scaling Multi-Task Bimanual Manipulation with Sparse Language-Conditioned Mixture-of-Experts Transformers [3.890941830250993]
  We propose a lightweight multi-task imitation learning framework for bimanual manipulation.<n>MoE-ACT integrates sparse Mixture-of-Experts (MoE) modules into the Transformer encoder of ACT.<n>We show that MoE-ACT substantially improves multi-task performance.
  arXiv  Detail & Related papers  (2026-03-16T13:33:59Z)
• Beyond Quantity: Trajectory Diversity Scaling for Code Agents [51.71414642763219]
  Trajectory Diversity Scaling is a data synthesis framework for code agents that scales performance through diversity rather than raw volume.<n> TDScaling integrates four innovations: (1) a Business Cluster mechanism that captures real-service logical dependencies; (2) a blueprint-driven multi-agent paradigm that enforces trajectory coherence; and (3) an adaptive evolution mechanism that steers toward long-tail scenarios.
  arXiv  Detail & Related papers  (2026-02-03T07:43:03Z)
• RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering [62.63376387138257]
  We propose a plug-and-play intervention framework that adaptively steers large language models (LLMs) reasoning in activation space.<n>RISER constructs a library of reusable reasoning vectors and employs a lightweight Router to dynamically compose them for each input.<n>The Router is optimized via reinforcement learning under task-level rewards, activating latent cognitive primitives in an emergent and compositional manner.
  arXiv  Detail & Related papers  (2026-01-14T08:04:33Z)
• Heuristic Transformer: Belief Augmented In-Context Reinforcement Learning [1.8791091507292152]
  Heuristic Transformer (HT) is an in-context reinforcement learning approach that augments the in-context dataset with a belief distribution over rewards to achieve better decision-making.<n>We show that HT consistently surpasses comparable baselines in terms of both effectiveness and generalization.<n>Our method presents a promising direction to bridge the gap between belief-based augmentations and transformer-based decision-making.
  arXiv  Detail & Related papers  (2025-11-13T12:32:36Z)
• Mental Accounts for Actions: EWA-Inspired Attention in Decision Transformers [2.9385229328767983]
  We propose Experience-Weighted Attraction with Vector Quantization for Online Decision Transformers (EWA-VQ-ODT)<n>EWA-VQ-ODT is a lightweight module that maintains per-action mental accounts summarizing recent successes and failures.<n>On standard continuous-control benchmarks, EWA-VQ-ODT improves sample efficiency and average return over ODT.
  arXiv  Detail & Related papers  (2025-09-19T00:33:22Z)
• Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence [131.41894248194995]
  We propose context-oriented decomposition adaptation (CorDA), a novel method that initializes adapters in a task-aware manner.<n>Thanks to the task awareness, our method enables two optional adaptation modes, knowledge-preserved mode (KPM) and instruction-previewed mode (IPM)
  arXiv  Detail & Related papers  (2025-06-16T07:55:14Z)
• ReaCritic: Large Reasoning Transformer-based DRL Critic-model Scaling For Heterogeneous Networks [4.931691794637798]
  Heterogeneous Networks (HetNets) pose critical challenges for intelligent management due to the diverse user requirements and time-varying wireless conditions.<n>We propose ReaCritic, a large reasoning transformer-based criticmodel scaling scheme that brings reasoning ability into Deep Reinforcement Learning.<n>It is compatible with a broad range of value-based and actor-critic DRL algorithms and enhances generalization in dynamic wireless environments.
  arXiv  Detail & Related papers  (2025-05-16T08:42:08Z)
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [73.75271615101754]
  We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.<n>Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.<n>Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
  arXiv  Detail & Related papers  (2025-03-25T15:19:56Z)
• Diffusion Transformer Policy [48.50988753948537]
  We propose a large multi-modal diffusion transformer, dubbed as Diffusion Transformer Policy, to model continuous end-effector actions.<n>By leveraging the scaling capability of transformers, the proposed approach can effectively model continuous end-effector actions across large diverse robot datasets.
  arXiv  Detail & Related papers  (2024-10-21T12:43:54Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.