Ordered Action Tokens for Visuomotor Policy Learning

• URL: http://arxiv.org/abs/2607.21670v1
• Date: Thu, 23 Jul 2026 07:04:51 GMT
• Title: Ordered Action Tokens for Visuomotor Policy Learning
• Abstract summary: Action tokenization maps continuous robot action chunks to discrete tokens.<n>We introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies high compression, total decodability, and an ordered token space.<n>OAT consistently delivers strong policy performance while offering significantly greater flexibility at inference time.
• Score: 47.37773778249323
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Action tokenization maps continuous robot action chunks to discrete tokens and has become an important interface for modern visuomotor policies. Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences or learned latent tokenizers that lack structure, limiting their compatibility with downstream policies. In this work, we identify three desiderata for action tokenization - high compression, total decodability, and an ordered token space - and introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies all three. OAT discretizes action chunks into an ordered sequence of tokens using a transformer with registers, finite scalar quantization, and ordering-inducing training mechanisms. By training each token prefix to decode into a valid action chunk, OAT places coarse control information in early tokens and uses later tokens to refine residual detail, yielding an anytime tradeoff between inference cost and action fidelity. We validate OAT in two prevailing uses of action tokens: autoregressive policies that generate tokens for control, and token co-training policies that use token losses to shape the vision-language model context consumed by a flow-based action expert. Across three policy backbones and more than 60 tasks spanning five simulation benchmarks and real-world settings, OAT consistently delivers strong policy performance while offering significantly greater flexibility at inference time.

Related papers

• RUTA: Principled Visual Token Allocation via Rate-Utility Optimization [39.416519746709305]
  High-resolution images and long videos provide vision-language models with rich context for multimodal reasoning and fine-grained perception.<n>We introduce RUTA, a principled Rate-Utility Token Allocation method that performs pre-LLM reduction by jointly learning which tokens to retain.<n>RUTA is optimized with a penalized rate-utility objective that balances downstream task loss against expected token usage.
  arXiv  Detail & Related papers  (2026-08-04T18:37:26Z)
• Resolving Action Bottleneck: Agentic Reinforcement Learning Informed by Token-Level Energy [64.72195169444738]
  Policy-gradient methods treat each token in a trajectory equally, leading to uniform credit assignment.<n>We show that such uniform credit assignment largely misallocates token-level training signals.<n>We propose an embarrassingly simple token reweighting approach, ActFocus, that downweights gradients on reasoning tokens.
  arXiv  Detail & Related papers  (2026-05-14T08:33:02Z)
• Bottleneck Tokens for Unified Multimodal Retrieval [16.707536543758344]
  Adapting decoder-only multimodal large language models (MLLMs) for unified multimodal retrieval faces two structural gaps.<n>First, existing methods rely on implicit pooling, which overloads the hidden state of a standard vocabulary token as the sequence-level representation.<n>Second, contrastive fine-tuning specifies what the embedding should match but provides no token-level guidance on how information should be compressed into it.<n>We introduce Bottleneck Tokens (BToks), a small set of learnable tokens that serve as a fixed-capacity explicit pooling mechanism.
  arXiv  Detail & Related papers  (2026-04-13T07:12:12Z)
• Drift-Aware Continual Tokenization for Generative Recommendation [44.13013639783267]
  Generative recommendation commonly adopts a two-stage pipeline in which a learnable tokenizer maps items to discrete token sequences.<n>Recent tokenizers further incorporate collaborative signals so that items with similar user-behavior patterns receive similar codes.<n>New items cause collision and shifts, while new interactions induce collaborative drift in existing items.<n>We propose DACT, a Drift-Aware Continual Tokenization framework with two stages: tokenizer fine-tuning and item-level drift confidence.
  arXiv  Detail & Related papers  (2026-03-31T13:02:47Z)
• OAT: Ordered Action Tokenization [44.20363344414952]
  Autoregressive policies offer a compelling foundation for scalable robot learning by enabling discrete abstraction, token-level reasoning, and flexible inference.<n>Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences, or learned latent tokenizers that lack structure.<n>In this work, we identify three desiderata for action tokenization - high compression, total decodability, and a left-to-right causally ordered token space - and introduce Ordered Action Tokenization (OAT)<n>OAT discretizes action chunks into an ordered sequence of tokens using transformer with registers, finite scalar quant
  arXiv  Detail & Related papers  (2026-02-04T05:01:04Z)
• Behavior-Equivalent Token: Single-Token Replacement for Long Prompts in LLMs [55.827877498548965]
  We propose a lightweight training framework that learns a single prompt-specific Behavior-Equivalent token ([BE])<n>The framework first trains [BE] to encode the natural-language content of the original system prompt via reconstruction, and then distills the prompt's downstream behavior into this single token.<n> Empirical evaluations on three datasets show that a single [BE] token achieves up to a 3000x reduction in prompt length, while retaining about 98% of the downstream performance of the original system prompts.
  arXiv  Detail & Related papers  (2025-11-28T15:22:52Z)
• BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning [24.858548048614878]
  We present the B-spline Encoded Action Sequence Tokenizer (BEAST)<n>BEAST encodes action sequences into compact discrete or continuous tokens using B-splines.<n>We evaluate BEAST across three established benchmarks consisting of 166 simulated tasks and on three distinct robot settings with a total of 8 real-world tasks.
  arXiv  Detail & Related papers  (2025-06-06T13:26:16Z)
• Enhancing Item Tokenization for Generative Recommendation through Self-Improvement [67.94240423434944]
  Generative recommendation systems are driven by large language models (LLMs)<n>Current item tokenization methods include using text descriptions, numerical strings, or sequences of discrete tokens.<n>We propose a self-improving item tokenization method that allows the LLM to refine its own item tokenizations during training process.
  arXiv  Detail & Related papers  (2024-12-22T21:56:15Z)
• SEP: Self-Enhanced Prompt Tuning for Visual-Language Model [93.94454894142413]
  We introduce a novel approach named Self-Enhanced Prompt Tuning (SEP) SEP explicitly incorporates discriminative prior knowledge to enhance both textual-level and visual-level embeddings. Comprehensive evaluations across various benchmarks and tasks confirm SEP's efficacy in prompt tuning.
  arXiv  Detail & Related papers  (2024-05-24T13:35:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.