WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation

• URL: http://arxiv.org/abs/2606.28320v1
• Date: Fri, 26 Jun 2026 17:58:06 GMT
• Title: WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation
• Abstract summary: We present WARP (Warp-Augmented Relative Progress), a novel fully self-supervised algorithm for learning dense, signed relative progress magnitudes.<n> WARP generates per-frame progress targets via time-warp augmentations of demonstrations and we train WARP-RM to predict the normalized elapsed time between input frames.<n>We then introduce WARP-BC, which leverages these scalar reward estimates to upweight high-level action chunks during behavior cloning.
• Score: 16.464426150252255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling imitation learning requires large datasets, yet human teleoperation inevitably produces mixed-quality demonstrations containing hesitations and recoveries. Prior frame-level progress reward models supervise on absolute temporal progress proxies that suffer from label noise, or require costly human annotations to define subtask boundaries. We present WARP (Warp-Augmented Relative Progress), a novel fully self-supervised algorithm for learning dense, signed relative progress magnitudes directly from successful demonstrations. WARP generates per-frame progress targets via time-warp augmentations of demonstrations (variable playback speeds and reversals) and we train WARP-RM to predict the normalized elapsed time between input frames. Aggregating these predictions across overlapping windows yields a dense frame-level progress signal. We then introduce WARP-BC, which leverages these scalar reward estimates to upweight high-advantage action chunks during behavior cloning, where chunk-level advantage is obtained by aggregating per-frame rewards. We evaluate our approach on a physical bimanual robot system performing a long-horizon deformable object manipulation task: folding T-shirts from a random crumpled start. To evaluate policy robustness against suboptimal data, we construct training datasets of varying quality using episode length as a proxy for teleoperation sub-optimality. As the dataset is widened to admit more inefficiencies, WARP-BC maintains a 19/20 success rate compared to vanilla BC's collapse to 2/20, improving throughput by up to 18x.

Related papers

• Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning [13.544618187879296]
  emphProgressive Seed Pruning (PSP) scores intermediate denoised estimates and progressively narrows the set so that only promising trajectories are fully denoised.<n>PSP consistently improves reward-guided selection and achieves higher GenEval scores (automated) and better human evaluation on prompt-alignment.
  arXiv  Detail & Related papers  (2026-07-23T17:59:46Z)
• Rank-Then-Act: Reward-Free Control from Frame-Order Progress [5.357451930622806]
  Rank-Then-Act (RTA) is a framework for learning control policies from expert video demonstrations without environment rewards.<n>We train a Vision-Language Model (VLM) offline as a progress-based ordinal scorer.<n>We compute the Spearman rank correlation between predicted progress rankings and true temporal indices, yielding a bounded, scale-invariant learning signal.
  arXiv  Detail & Related papers  (2026-07-02T08:50:32Z)
• RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents [13.362388357375082]
  Multi-turn tool-use RL is bottlenecked by the rapid depletion of informative samples in static datasets.<n>We propose RODS (Reward-driven Online Data Synthesis) to resolve this depletion.
  arXiv  Detail & Related papers  (2026-06-17T13:13:32Z)
• DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners? [57.585275546688116]
  Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents.<n>We argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world.<n>We introduce DIRECT, a routing framework that uses multimodal scene context to allocate compute per prompt.
  arXiv  Detail & Related papers  (2026-06-10T17:58:49Z)
• TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation [61.35569005726248]
  Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size.<n>Non-deterministic generative models, such as diffusion and autoregressive models, have been widely adopted in the field of embodied control.<n>We propose textbfTapSampling, a plug-and-play framework for inference-time sampling.
  arXiv  Detail & Related papers  (2026-05-25T08:03:31Z)
• ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation [57.07494675832939]
  Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness.<n>In this work, we investigate the estimation and integration of task progress, proposing a novel model named textbf vla<n>Experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates.
  arXiv  Detail & Related papers  (2026-03-29T12:38:11Z)
• OmniSAT: Compact Action Token, Faster Auto Regression [70.70037017501357]
  We introduce an Omni Swift Action Tokenizer, which learns a compact, transferable action representation.<n>The resulting discrete tokenization shortens the training sequence by 6.8$times$, and lowers the target entropy.
  arXiv  Detail & Related papers  (2025-10-08T03:55:24Z)
• Streaming Looking Ahead with Token-level Self-reward [50.699168440048716]
  We propose a policy model with token-level self-reward modeling (TRM) capability to eliminate the need for external models and extra communication.<n>In addition, we propose a streaming-looking-ahead (SLA) algorithm to further boost search efficiency with better parallelization.<n>If we combine SLA with reinforcement fine-tuning techniques such as DPO, SLA achieves an overall win rate of 89.4%.
  arXiv  Detail & Related papers  (2025-02-24T22:35:53Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Post-Processing Temporal Action Detection [134.26292288193298]
  Temporal Action Detection (TAD) methods typically take a pre-processing step in converting an input varying-length video into a fixed-length snippet representation sequence. This pre-processing step would temporally downsample the video, reducing the inference resolution and hampering the detection performance in the original temporal resolution. We introduce a novel model-agnostic post-processing method without model redesign and retraining.
  arXiv  Detail & Related papers  (2022-11-27T19:50:37Z)
• Large Scale Time-Series Representation Learning via Simultaneous Low and High Frequency Feature Bootstrapping [7.0064929761691745]
  We propose a non-contrastive self-supervised learning approach efficiently captures low and high-frequency time-varying features. Our method takes raw time series data as input and creates two different augmented views for two branches of the model. To demonstrate the robustness of our model we performed extensive experiments and ablation studies on five real-world time-series datasets.
  arXiv  Detail & Related papers  (2022-04-24T14:39:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.