SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World Models

• URL: http://arxiv.org/abs/2606.00664v1
• Date: Sat, 30 May 2026 10:41:34 GMT
• Title: SKIP: Sparse Keyframe Interpolation Paradigm for Efficient Embodied World Models
• Abstract summary: We propose Sparse Keyframe Interpolation Paradigm (SKIP), an event-preserving sparse-to-dense framework that avoids dense frame-by-frame generation.<n>SKIP identifies task-relevants by leveraging robot-aware multimodal features.<n>It then synthesizes only these Ems with a sparse video diffusion model.<n>A learned gap predictor videos and an action-conditioned interpolator subsequently reconstruct the missing intervals according to the robot actions.
• Score: 34.49867155987541
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied world models have emerged as a promising paradigm in robotics by predicting how robot actions affect the surrounding scene. However, the rollout inference remains computationally expensive in pixel space, as long-horizon manipulation videos typically have to be generated frame by frame. This cost cannot be easily reduced by indiscriminately dropping frames, since downstream policies rely on complete preservation of sparse task-relevant events such as approach, contact, grasp, and release. To address this challenge, we propose Sparse Keyframe Interpolation Paradigm (SKIP), an event-preserving sparse-to-dense framework that avoids dense frame-by-frame generation. SKIP first identifies task-relevant keyframes by leveraging robot-aware multimodal features. It then synthesizes only these keyframes with a sparse video diffusion model. A learned gap predictor and an action-conditioned interpolator subsequently reconstruct the missing intervals according to the robot actions. On LIBERO, SKIP generates dense rollouts $4.16\times$ faster than a dense baseline while improving visual fidelity and reducing aggregate FVD by $89.0\%$. Importantly, SKIP-generated videos are effective policy-training data. Even when they fully replace real demonstrations, $π_{0.5}$ success drops only $1.3$ pp in LIBERO simulation and $6.7$ pp on the real robot, whereas fully dense frame-by-frame generation collapses by $48$ to $58$ pp.

Related papers

• StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models [84.17128030384099]
  StreamPI is a streaming multimodal temporal modeling framework.<n>It equips single-frame VLA with temporal reasoning capability without introducing any additional parameters.<n> Experiments on real-robot tasks spanning memory-dependent and precise perception scenarios, as well as the simulation benchmark LIBERO, demonstrate that StreamPI outperforms pi0.5 across diverse tasks.
  arXiv  Detail & Related papers  (2026-08-26T17:33:19Z)
• PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models [63.31550146180189]
  This paper introduces a training-free $mathbfP$ersistence-Aware $mathbfC$ompression and $mathbfA$ggregation (PCA) method to preserve high-sampling raw visual information before the encoding stage.<n>PCA consistently outperforms existing state-of-the-art approaches in both efficiency and accuracy.
  arXiv  Detail & Related papers  (2026-07-22T05:17:38Z)
• Fast Enough to Act: Spatio-Temporal Visual Token Merging for Low-Latency Robotic VLMs and VLAs [21.02858354959528]
  Vision-language models and vision-language action models endow the robot with unprecedented capabilities.<n>The input of video and high-resolution images yields a massive number of visual tokens, leading to extremely high inference latency and severely hindering the robot's real-time control.<n>We propose ST-Merge, a plug-and-play framework that efficiently fuses redundant tokens directly during the visual encoding phase.
  arXiv  Detail & Related papers  (2026-06-28T11:42:55Z)
• Diffusion Transformer World-Action Model for AV Scene Prediction [0.0]
  Action-conditioned world models let an autonomous vehicle predict future camera scenes from its own planned controls.<n>But at compact, trainable scale the futures are ambiguous and the field's standard distortion metrics actively mislead.<n>We confront this with a compact latent world model that predicts future scene latents a frozen decoder renders to $256 times 256$ frames up to 8 seconds ahead.
  arXiv  Detail & Related papers  (2026-06-11T07:24:12Z)
• RobotPan: A 360$^\circ$ Surround-View Robotic Vision System for Embodied Perception [47.76543396190029]
  We introduce a surround-view robotic vision system that combines six cameras with LiDAR to provide full 360$circ$ visual coverage.<n>We also present textscRobotPan, a feed-forward framework that predicts emphmetric-scaled and emphcompact 3D Gaussians from calibrated sparse-view inputs.<n>Experiments show that textscRobotPan achieves competitive quality against prior feed-forward reconstruction and view-synthesis methods.
  arXiv  Detail & Related papers  (2026-04-15T04:58:23Z)
• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control [16.562259973551786]
  We introduce DiT4DiT, an end-to-end Video-Action Model that couples a video Diffusion Transformer with an action Diffusion Transformer.<n>Instead of relying on reconstructed future frames, DiT4DiT extracts intermediate denoising features from the video generation process.<n>It achieves state-of-the-art results, reaching average success rates of 98.6% on LIBERO and 50.8% on RoboCasa GR1.
  arXiv  Detail & Related papers  (2026-03-11T06:03:53Z)
• An Efficient Quality Metric for Video Frame Interpolation Based on Motion-Field Divergence [0.3823356975862005]
  Video frame is a fundamental tool for temporal video enhancement, but existing quality metrics struggle to evaluate the impact of artefacts effectively.<n>We present $textPSNR_textDIV$, a novel full-reference quality metric that enhances PSNR through motion divergence weighting.<n>Our approach highlights singularities in motion fields which is then used to weight image errors.
  arXiv  Detail & Related papers  (2025-10-01T18:40:38Z)
• H$_{2}$OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers [124.11648300910444]
  We present a hierarchical plug-and-play pruning-and-$-recovering framework, called Hierarchical Hourglass Tokenizer (H$_2$OT)<n>Our method is general-purpose: it can be easily incorporated into common VPT models on both seq2seq and seq2frame pipelines.
  arXiv  Detail & Related papers  (2025-09-08T17:59:59Z)
• FLARE: Robot Learning with Implicit World Modeling [87.81846091038676]
  $textbfFLARE$ integrates predictive latent world modeling into robot policy learning.<n>$textbfFLARE$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%.<n>Our results establish $textbfFLARE$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.
  arXiv  Detail & Related papers  (2025-05-21T15:33:27Z)
• Towards Scalable Modeling of Compressed Videos for Efficient Action Recognition [6.168286187549952]
  We propose a hybrid end-to-end framework that factorizes learning across three key concepts to reduce inference cost by $330times$ versus prior art.<n> Experiments show that our method results in a lightweight architecture achieving state-of-the-art video recognition performance.
  arXiv  Detail & Related papers  (2025-03-17T21:13:48Z)
• IBVC: Interpolation-driven B-frame Video Compression [68.18440522300536]
  B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction. Previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation. We propose a simple yet effective structure called Interpolation-B-frame Video Compression (IBVC) to address these issues.
  arXiv  Detail & Related papers  (2023-09-25T02:45:51Z)
• All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modeling [52.425236515695914]
  State-of-the-art methods are iterative solutions interpolating one frame at the time. This work introduces a true multi-frame interpolator. It utilizes a pyramidal style network in the temporal domain to complete the multi-frame task in one-shot.
  arXiv  Detail & Related papers  (2020-07-23T02:34:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.