Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer

• URL: http://arxiv.org/abs/2605.06104v1
• Date: Thu, 07 May 2026 12:20:25 GMT
• Title: Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer
• Abstract summary: Decision Transformer (DT) formulates offline reinforcement learning as autoregressive sequence modeling.<n>We propose SlimDT, which removes Return-to-Go (RTG) from the autoregressive sequence.<n>On the D4RL benchmark, SlimDT surpasses standard DT across various tasks and achieves performance comparable to existing state-of-the-art methods.
• Score: 6.196814138255951
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decision Transformer (DT) formulates offline reinforcement learning as autoregressive sequence modeling, achieving promising results by predicting actions from a sequence of Return-to-Go (RTG), state, and action tokens. However, RTG is a scalar that summarizes future rewards, containing far less information than typical state or action vectors, yet it consumes the same computational budget per token. Worse, the self-attention cost of Transformers grows quadratically with sequence length, so including RTG as a separate token adds unnecessary overhead. We propose SlimDT, which removes RTG from the autoregressive sequence. Instead, we inject RTG information into the state representations before the sequential modeling step, allowing the Transformer to process only a compact (state, action) sequence. This reduces the sequence length by one-third, directly improving inference efficiency. On the D4RL benchmark, SlimDT surpasses standard DT across various tasks and achieves performance comparable to existing state-of-the-art methods. Decoupling a sparse conditioning signal from an information-rich sequence thus yields both computational gains and higher task performance.

Related papers

• Decoupling Return-to-Go for Efficient Decision Transformer [6.429850804144503]
  Decision Transformer (DT) has established a powerful sequence modeling approach to offline reinforcement learning.<n>Decoupled DT (DDT) simplifies the architecture by processing only observation and action sequences through the Transformer.<n>Our experiments show that DDT significantly outperforms DT and establishes competitive performance against state-of-the-art DT variants.
  arXiv  Detail & Related papers  (2026-01-22T13:42:08Z)
• OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender [32.265739328468584]
  OneTrans is a unified Transformer backbone that simultaneously performs user-behavior sequence modeling and feature interaction.<n>We show that OneTrans scales efficiently with increasing parameters, consistently outperforms strong baselines, and yields a 5.68% lift in per-user GMV in online A/B tests.
  arXiv  Detail & Related papers  (2025-10-30T03:30:12Z)
• Segmented Recurrent Transformer: An Efficient Sequence-to-Sequence Model [10.473819332984005]
  We propose a segmented recurrent transformer (SRformer) that combines segmented (local) attention with recurrent attention. The proposed model achieves $6-22%$ higher ROUGE1 scores than a segmented transformer and outperforms other recurrent transformer approaches.
  arXiv  Detail & Related papers  (2023-05-24T03:47:22Z)
• Optimizing Non-Autoregressive Transformers with Contrastive Learning [74.46714706658517]
  Non-autoregressive Transformers (NATs) reduce the inference latency of Autoregressive Transformers (ATs) by predicting words all at once rather than in sequential order. In this paper, we propose to ease the difficulty of modality learning via sampling from the model distribution instead of the data distribution.
  arXiv  Detail & Related papers  (2023-05-23T04:20:13Z)
• Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition [62.83832841523525]
  We propose a fast and accurate parallel transformer, termed Paraformer. It accurately predicts the number of output tokens and extract hidden variables. It can attain comparable performance to the state-of-the-art AR transformer, with more than 10x speedup.
  arXiv  Detail & Related papers  (2022-06-16T17:24:14Z)
• Directed Acyclic Transformer for Non-Autoregressive Machine Translation [93.31114105366461]
  Directed Acyclic Transfomer (DA-Transformer) represents hidden states in a Directed Acyclic Graph (DAG) DA-Transformer substantially outperforms previous NATs by about 3 BLEU on average.
  arXiv  Detail & Related papers  (2022-05-16T06:02:29Z)
• Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation [109.46348908829697]
  We propose a novel Edit-Invariant Sequence Loss (EISL), which computes the matching loss of a target n-gram with all n-grams in the generated sequence. We conduct experiments on three tasks: machine translation with noisy target sequences, unsupervised text style transfer, and non-autoregressive machine translation.
  arXiv  Detail & Related papers  (2021-06-29T03:59:21Z)
• IOT: Instance-wise Layer Reordering for Transformer Structures [173.39918590438245]
  We break the assumption of the fixed layer order in the Transformer and introduce instance-wise layer reordering into the model structure. Our method can also be applied to other architectures beyond Transformer.
  arXiv  Detail & Related papers  (2021-03-05T03:44:42Z)
• Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition [66.47000813920617]
  We propose a spike-triggered non-autoregressive transformer model for end-to-end speech recognition. The proposed model can accurately predict the length of the target sequence and achieve a competitive performance. The model even achieves a real-time factor of 0.0056, which exceeds all mainstream speech recognition models.
  arXiv  Detail & Related papers  (2020-05-16T08:27:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.