Mimic Intent, Not Just Trajectories

• URL: http://arxiv.org/abs/2602.08602v2
• Date: Wed, 18 Mar 2026 08:05:39 GMT
• Title: Mimic Intent, Not Just Trajectories
• Abstract summary: We argue this stems from mimicking raw trajectories without understanding the underlying intent.<n>We propose explicitly disentangling behavior intent from execution details in end-2-end IL: Mimic Intent, Not just Trajectories.<n> Experiments on several manipulation benchmarks and on a real robot demonstrate state-of-the-art success rates, superior inference efficiency, robust generalization against disturbances, and effective one-shot transfer.
• Score: 39.77112205526461
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While imitation learning (IL) has achieved impressive success in dexterous manipulation through generative modeling and pretraining, state-of-the-art approaches like Vision-Language-Action (VLA) models still struggle with adaptation to environmental changes and skill transfer. We argue this stems from mimicking raw trajectories without understanding the underlying intent. To address this, we propose explicitly disentangling behavior intent from execution details in end-2-end IL: Mimic Intent, Not just Trajectories(MINT). We achieve this via multi-scale frequency-space tokenization, which enforces a spectral decomposition of action chunk representation. We learn action tokens with a multi-scale coarse-to-fine structure, and force the coarsest token to capture low-frequency global structure and finer tokens to encode high-frequency details. This yields an abstract Intent token that facilitates planning and transfer, and multi-scale Execution tokens that enable precise adaptation to environmental dynamics. Building on this hierarchy, our policy generates trajectories through next-scale autoregression, performing progressive intent-to-execution reasoning, thus boosting learning efficiency and generalization. Crucially, this disentanglement enables one-shot transfer of skills, by simply injecting the Intent token from a demonstration into the autoregressive generation process. Experiments on several manipulation benchmarks and on a real robot demonstrate state-of-the-art success rates, superior inference efficiency, robust generalization against disturbances, and effective one-shot transfer.

Related papers

• Resolving Action Bottleneck: Agentic Reinforcement Learning Informed by Token-Level Energy [64.72195169444738]
  Policy-gradient methods treat each token in a trajectory equally, leading to uniform credit assignment.<n>We show that such uniform credit assignment largely misallocates token-level training signals.<n>We propose an embarrassingly simple token reweighting approach, ActFocus, that downweights gradients on reasoning tokens.
  arXiv  Detail & Related papers  (2026-05-14T08:33:02Z)
• OAT: Ordered Action Tokenization [44.20363344414952]
  Autoregressive policies offer a compelling foundation for scalable robot learning by enabling discrete abstraction, token-level reasoning, and flexible inference.<n>Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences, or learned latent tokenizers that lack structure.<n>In this work, we identify three desiderata for action tokenization - high compression, total decodability, and a left-to-right causally ordered token space - and introduce Ordered Action Tokenization (OAT)<n>OAT discretizes action chunks into an ordered sequence of tokens using transformer with registers, finite scalar quant
  arXiv  Detail & Related papers  (2026-02-04T05:01:04Z)
• BiKC+: Bimanual Hierarchical Imitation with Keypose-Conditioned Coordination-Aware Consistency Policies [25.25845809170421]
  We propose a novel keypose-conditioned coordination-aware consistency policy tailored for bimanual manipulation.<n>Our framework instantiates hierarchical imitation learning with a high-level keypose predictor and a low-level trajectory generator.<n>In particular, we devise an innovative approach for identifying bimanual keyposes, considering both robot-centric action features and task-centric operation styles.
  arXiv  Detail & Related papers  (2026-01-17T17:33:18Z)
• Bridging Scale Discrepancies in Robotic Control via Language-Based Action Representations [20.612898908232907]
  We propose a semantically grounded linguistic representation to normalize actions for efficient pretraining.<n>We show that the proposed method significantly improves generalization and transferability in robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-09T12:45:12Z)
• ActionSink: Toward Precise Robot Manipulation with Dynamic Integration of Action Flow [93.00917887667234]
  This paper introduces a novel robot manipulation framework, i.e., ActionSink, to pave the way toward precise action estimations.<n>As the name suggests, ActionSink reformulates the actions of robots as action-caused optical flows from videos, called "action flow"<n>Our framework outperformed prior SOTA on the LIBERO benchmark by a 7.9% success rate, and obtained nearly an 8% accuracy gain on the challenging long-horizon visual task LIBERO-Long.
  arXiv  Detail & Related papers  (2025-08-05T08:46:17Z)
• FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens [47.735852718586216]
  We propose a novel paradigm for visuomotor policy learning that progressively models hierarchical frequency components.<n>To further enhance precision, we introduce continuous latent representations that maintain smoothness and continuity in the action space.<n>Our approach outperforms existing methods in both accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-06-02T12:13:51Z)
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [73.75271615101754]
  We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.<n>Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.<n>Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
  arXiv  Detail & Related papers  (2025-03-25T15:19:56Z)
• BiKC: Keypose-Conditioned Consistency Policy for Bimanual Robotic Manipulation [48.08416841005715]
  We introduce a novel keypose-conditioned consistency policy tailored for bimanual manipulation. It is a hierarchical imitation learning framework that consists of a high-level keypose predictor and a low-level trajectory generator. Simulated and real-world experimental results demonstrate that the proposed approach surpasses baseline methods in terms of success rate and operational efficiency.
  arXiv  Detail & Related papers  (2024-06-14T14:49:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.