Learning High-Frequency Continuous Action Chunks in Latent Space

• URL: http://arxiv.org/abs/2605.24931v1
• Date: Sun, 24 May 2026 08:22:14 GMT
• Title: Learning High-Frequency Continuous Action Chunks in Latent Space
• Abstract summary: Modern robotic policies increasingly rely on action chunking to execute complex tasks in the physical world.<n>At such high frequencies, policies often fail to generate actions that are both temporally smooth and spatially consistent.<n>We address this challenge by shifting high-frequency action learning from the action space to a latent space with variational autoencoder (VAE)<n>This formulation significantly improves both temporal and spatial consistency of high-frequency control.
• Score: 10.321199672785392
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern robotic policies increasingly rely on action chunking to execute complex tasks in the physical world. While action chunking improves temporal consistency at moderate action frequencies, it becomes insufficient when the action frequency is further increased (e.g., to 60~Hz). At such high frequencies, policies often fail to generate actions that are both temporally smooth and spatially consistent. We address this challenge by shifting high-frequency action learning from the action space to a latent space with variational autoencoder (VAE). This formulation significantly improves both temporal and spatial consistency of high-frequency control. To enable smooth real-time execution, we further introduce Reuse-then-Refine, a chunk-level refine strategy that improves continuity between adjacent action chunks under asynchronous inference. As a result, robots controlled by our policy can execute complex contact-rich tasks continuously, with less pauses and jerky motions. Experiments on three real-world contact-rich robotic tasks show that our approach consistently completes tasks with smooth motions. Our code and data are available at https://github.com/tars-robotics/RTR.

Related papers

• Trajectory-Level Continuous Action Representation for Robotic Manipulation [31.92437041447019]
  Existing visuomotor systems often entangle action representation with control frequency or rely on fixed temporal parameterizations.<n>We propose CAT, a trajectory-level continuous action representation framework for robotic manipulation.<n>Cat encodes action trajectories within a fixed real-time interval into a set of continuous latent tokens.
  arXiv  Detail & Related papers  (2026-08-25T06:19:00Z)
• Frequency-Aware Flow Matching for Continuous and Consistent Robotic Action Generation [46.998628831231194]
  Existing methods rely on discretized action chunks, making them brittle to demonstrations collected at heterogeneous control.<n>We propose Frequency-Aware Flow Matching (FAFM), which outputs continuous, temporally consistent actions.<n>FAFM improves success rates, multimodal expressivity, motion smoothness, convergence speed, robustness to mechanical bias and mixed-frequency input.
  arXiv  Detail & Related papers  (2026-06-18T11:58:30Z)
• HiPolicy: Hierarchical Multi-Frequency Action Chunking for Policy Learning [17.3424491065084]
  HiPolicy is a hierarchical multi-frequency action chunking framework that predicts action sequences at different frequencies to capture both coarse high-level plans and precise reactive motions.<n> Experiments on diverse simulated benchmarks and real-world manipulation tasks show that HiPolicy can be seamlessly integrated into existing 2D and 3D generative policies.
  arXiv  Detail & Related papers  (2026-04-07T16:47:38Z)
• Enabling Dynamic Tracking in Vision-Language-Action Models via Time-Discrete and Time-Continuous Velocity Feedforward [11.066720921275648]
  Vision-language-action (VLA) models have shown great promise for robot manipulation.<n>Their deployment on rigid industrial robots remains challenging due to the inherent trade-off between compliance and responsiveness.<n>This paper shows the importance of integrating velocity feedforward terms into VLA policies to resolve this trade-off.
  arXiv  Detail & Related papers  (2026-03-17T07:50:00Z)
• Closed-Loop Action Chunks with Dynamic Corrections for Training-Free Diffusion Policy [52.106797722292896]
  We present DCDP, a Dynamic Closed-Loop Diffusion Policy framework that integrates chunk-based action generation with real-time correction.<n>In dynamic PushT simulations, DCDP improves adaptability by 19% without retraining while requiring only 5% additional computation.
  arXiv  Detail & Related papers  (2026-03-02T15:04:18Z)
• Real-Time Robot Execution with Masked Action Chunking [38.37108371991901]
  Real-time execution is essential for cyber-physical systems such as robots.<n>Asynchronous inference has recently emerged as a system-level paradigm for real-time robot manipulation.<n>We propose REMAC, which learns corrective adjustments on the pretrained policy through masked action chunking.
  arXiv  Detail & Related papers  (2026-01-27T23:48:32Z)
• ImplicitRDP: An End-to-End Visual-Force Diffusion Policy with Structural Slow-Fast Learning [52.86018040861575]
  We propose a unified end-to-end visual-force diffusion policy that integrates visual planning and reactive force control within a single network.<n>We introduce Structural Slow-Fast Learning, a mechanism utilizing causal attention to simultaneously process asynchronous visual and force tokens.<n>Experiments on contact-rich tasks demonstrate that ImplicitRDP significantly outperforms both vision-only and hierarchical baselines.
  arXiv  Detail & Related papers  (2025-12-11T18:59:46Z)
• Real-Time Execution of Action Chunking Flow Policies [49.1574468325115]
  This paper presents a novel inference-time algorithm that enables asynchronous execution of action interacting systems.<n>It is applicable to any diffusion- or VLA-based systems executing out of the box with no re-training.<n>Results show that RTC is fast, performant, and uniquely robust to inference manipulation.
  arXiv  Detail & Related papers  (2025-06-09T01:01:59Z)
• FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens [47.735852718586216]
  We propose a novel paradigm for visuomotor policy learning that progressively models hierarchical frequency components.<n>To further enhance precision, we introduce continuous latent representations that maintain smoothness and continuity in the action space.<n>Our approach outperforms existing methods in both accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-06-02T12:13:51Z)
• Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation [58.95799126311524]
  Humans can accomplish contact-rich tasks using vision and touch, with highly reactive capabilities such as fast response to external changes and adaptive control of contact forces.<n>Existing visual imitation learning approaches rely on action chunking to model complex behaviors.<n>We introduce TactAR, a low-cost teleoperation system that provides real-time tactile feedback through Augmented Reality.
  arXiv  Detail & Related papers  (2025-03-04T18:58:21Z)
• Latent Exploration for Reinforcement Learning [87.42776741119653]
  In Reinforcement Learning, agents learn policies by exploring and interacting with the environment. We propose LATent TIme-Correlated Exploration (Lattice), a method to inject temporally-correlated noise into the latent state of the policy network.
  arXiv  Detail & Related papers  (2023-05-31T17:40:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.