E$^2$DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulation

• URL: http://arxiv.org/abs/2605.00159v1
• Date: Thu, 30 Apr 2026 19:28:44 GMT
• Title: E$^2$DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulation
• Abstract summary: Decision Transformer (DT) has emerged as an effective framework for addressing long-horizon tasks.<n>E$2$DT is a DT-guided k-Determinantal Point Process sampling framework.<n>Our framework is experience-aware, allowing E$2$DT to be both efficient.
• Score: 12.326967455610536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In reinforcement learning (RL) for robotic manipulation, the Decision Transformer (DT) has emerged as an effective framework for addressing long-horizon tasks. However, DT's performance depends heavily on the coverage of collected experiences. Without an active exploration mechanism, standard DT relies on uniform replay, which leads to poor sample efficiency, limited exploration, and reduced overall effectiveness. At the same time, while excessive exploration can help avoid local optima, it often delays policy convergence and leads to degraded efficiency. To address these limitations, we propose E$^2$DT, a DT-guided k-Determinantal Point Process sampling framework that enables the model to actively shape its own experience selection. Our framework is experience-aware, allowing E$^2$DT to be both efficient, by prioritizing sampling quality, such as high-return, high-uncertainty, and underrepresented trajectories, and effective, by ensuring diversity across trajectory windows to preserve policy optimality. Specifically, DT's internal latent embeddings measure diversity across trajectory windows, while quality is quantified through a composite metric that integrates return-to-go (RTG) quantiles, predictive uncertainty, and stage coverage based on inverse frequency. These two dimensions are integrated into a novel quality-diversity joint kernel that prioritizes the most informative experiences, thereby enabling learning that is both efficient and effective. We evaluate E$^2$DT on challenging robotic manipulation benchmarks in both simulation and real-robot settings. Results show that it consistently outperforms prior methods. These findings demonstrate that coupling policy learning with experience-aware sampling provides a principled path toward robust long-horizon robotic learning.

Related papers

• TSD: A Physics-Inspired Trajectory Saliency Detector for Efficient Imitation Learning [9.737683365003322]
  Trajectory Saliency Detector (TSD) is a training-free and plug-and-play framework to identify trajectory saliency.<n>TSD employs two physically-grounded metrics: spatial entropy to capture fine-grained manipulation and centripetal acceleration to detect agile maneuvering.<n>Extensive experiments in both simulation and real-world settings demonstrate that models trained on TSD-condensed datasets achieve comparable or even superior performance with 25% less data on average.
  arXiv  Detail & Related papers  (2026-06-22T14:06:04Z)
• Self-Imitated Diffusion Policy for Efficient and Robust Visual Navigation [7.341858898582114]
  Self-Imitated Diffusion Policy (SIDP) is a novel framework that learns improved planning by selectively imitating a set of trajectories sampled from itself.<n>Specifically, SIDP introduces a reward-guided self-imitation mechanism that encourages the policy to consistently produce high-quality trajectories efficiently.
  arXiv  Detail & Related papers  (2026-01-30T13:27:59Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• Large Language Model-Empowered Decision Transformer for UAV-Enabled Data Collection [71.84636717632206]
  Unmanned aerial vehicles (UAVs) for reliable and energy-efficient data collection from spatially distributed devices holds great promise in supporting Internet of Things (IoT) applications.<n>We propose a joint language model (LLM) to learn effective UAV control policies.<n>LLM-CRDT outperforms benchmark online and offline methods, achieving up to 36.7% higher energy efficiency than current state-of-the-art DT approaches.
  arXiv  Detail & Related papers  (2025-09-17T13:05:08Z)
• Is Diversity All You Need for Scalable Robotic Manipulation? [50.747150672933316]
  We investigate the nuanced role of data diversity in robot learning by examining three critical dimensions-task (what to do), embodiment (which robot to use), and expert (who demonstrates)-challenging the conventional intuition of "more diverse is better"<n>We show that task diversity proves more critical than per-task demonstration quantity, benefiting transfer from diverse pre-training tasks to novel downstream scenarios.<n>We propose a distribution debiasing method to mitigate velocity ambiguity, the yielding GO-1-Pro achieves substantial performance gains of 15%, equivalent to using 2.5 times pre-training data.
  arXiv  Detail & Related papers  (2025-07-08T17:52:44Z)
• Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning [64.88132446226474]
  A generalist agent must continuously learn and adapt throughout its lifetime, achieving efficient forward transfer while minimizing forgetting.<n>Previous work has explored parameter-efficient fine-tuning for single-task adaptation, effectively steering a frozen pretrained model with a small number of parameters.<n>We propose Dynamic Mixture of Progressive Efficient Expert Library (DMPEL) for lifelong robot learning.
  arXiv  Detail & Related papers  (2025-06-06T11:13:04Z)
• ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning [19.02836010747026]
  Decision Transformer (DT) generates actions conditioned on a desired future return. We propose to empower DT with dynamic programming to overcome DT's weaknesses. Our method demonstrates effective trajectory stitching and robust action generation in spite of the environmentality.
  arXiv  Detail & Related papers  (2023-09-12T02:05:43Z)
• Task-specific experimental design for treatment effect estimation [59.879567967089145]
  Large randomised trials (RCTs) are the standard for causal inference. Recent work has proposed more sample-efficient alternatives to RCTs, but these are not adaptable to the downstream application for which the causal effect is sought. We develop a task-specific approach to experimental design and derive sampling strategies customised to particular downstream applications.
  arXiv  Detail & Related papers  (2023-06-08T18:10:37Z)
• A Transferable and Automatic Tuning of Deep Reinforcement Learning for Cost Effective Phishing Detection [21.481974148873807]
  Many challenging real-world problems require the deployment of ensembles multiple complementary learning models. Deep Reinforcement Learning (DRL) offers a cost-effective alternative, where detectors are dynamically chosen based on the output of their predecessors.
  arXiv  Detail & Related papers  (2022-09-19T14:09:07Z)
• Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay [8.259694128526112]
  We propose diversity-based trajectory and goal selection with HER (DTGSH) We show that our method can learn more quickly and reach higher performance than other state-of-the-art approaches on all tasks.
  arXiv  Detail & Related papers  (2021-08-17T21:34:24Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.