RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Models

• URL: http://arxiv.org/abs/2605.03821v1
• Date: Tue, 05 May 2026 14:49:00 GMT
• Title: RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Models
• Abstract summary: We present RoboAlign-R1, a framework that combines reward-aligned post-training with stabilized long-horizon inference for robot video world models.<n>Results show that reward-aligned post-training and stabilized long-horizon decoding improve task consistency, physical realism, and long-horizon prediction quality.
• Score: 39.60337316193729
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing robot video world models are typically trained with low-level objectives such as reconstruction and perceptual similarity, which are poorly aligned with the capabilities that matter most for robot decision making, including instruction following, manipulation success, and physical plausibility. They also suffer from error accumulation in long-horizon autoregressive prediction. We present RoboAlign-R1, a framework that combines reward-aligned post-training with stabilized long-horizon inference for robot video world models. We construct RobotWorldBench, a benchmark of 10,000 annotated video-instruction pairs collected from four robot data sources, and train a multimodal teacher judge, RoboAlign-Judge, to provide fine-grained six-dimensional evaluation of generated videos. We then distill the teacher into a lightweight student reward model for efficient reinforcement-learning-based post-training. To reduce long-horizon rollout drift, we further introduce Sliding Window Re-encoding (SWR), a training-free inference strategy that periodically refreshes the generation context. Under our in-domain evaluation protocol, RoboAlign-R1 improves the aggregate six-dimension score by 10.1% over the strongest baseline, including gains of 7.5% on Manipulation Accuracy and 4.6% on Instruction Following; these ranking improvements are further supported by an external VLM-based cross-check and a blinded human study. Meanwhile, SWR improves long-horizon prediction quality with only about 1% additional latency, yielding a 2.8% gain in SSIM and a 9.8% reduction in LPIPS. Together, these results show that reward-aligned post-training and stabilized long-horizon decoding improve task consistency, physical realism, and long-horizon prediction quality in robot video world models.

Related papers

• Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories [36.86989207036016]
  Xiaomi-Robotics-1 is a vision-language-action (VLA) model for mobile manipulation.<n>It follows diverse language instructions to perform a wide range of mobile manipulation tasks.<n>It efficiently adapts to novel downstream tasks with minimal fine-tuning data.
  arXiv  Detail & Related papers  (2026-07-16T16:02:25Z)
• RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation [52.855830378221775]
  Video world models are emerging as a scalable alternative for evaluating generalist robot policies.<n>We introduce RoboWorld, an automated evaluation pipeline that pairs a fast video world model with a task-progress-aware vision-language model scoring.
  arXiv  Detail & Related papers  (2026-07-01T15:22:41Z)
• Persistent Robot World Models: Stabilizing Multi-Step Rollouts via Reinforcement Learning [18.397872306430006]
  Action-conditioned robot world models generate future video frames of the manipulated scene given a robot action sequence.<n>These models are optimized for short-term prediction and break down when deployed autoregressively.<n>We introduce a reinforcement learning scheme that trains the world model on its own autoregressive rollouts.
  arXiv  Detail & Related papers  (2026-03-26T17:36:08Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control [16.562259973551786]
  We introduce DiT4DiT, an end-to-end Video-Action Model that couples a video Diffusion Transformer with an action Diffusion Transformer.<n>Instead of relying on reconstructed future frames, DiT4DiT extracts intermediate denoising features from the video generation process.<n>It achieves state-of-the-art results, reaching average success rates of 98.6% on LIBERO and 50.8% on RoboCasa GR1.
  arXiv  Detail & Related papers  (2026-03-11T06:03:53Z)
• Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons [69.87766750714945]
  General-purpose robot reward models are typically trained to predict absolute task progress from expert demonstrations.<n>We introduce Robometer, a scalable reward modeling framework that combines intra-trajectory progress supervision with inter-trajectory preference supervision.<n>Robometer is trained with a dual objective: a frame-level progress loss that anchors reward magnitude on expert data, and a trajectory-comparison preference loss that imposes global ordering constraints.
  arXiv  Detail & Related papers  (2026-03-02T17:38:58Z)
• Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution [32.93468341343403]
  We introduce Xiaomi-Robotics-0, an advanced vision-language-action (VLA) model optimized for high performance and fast and smooth real-time execution.<n>Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, endowing it with broad and generalizable action-generation capabilities.<n>We evaluate Xiaomi-Robotics-0 extensively in simulation benchmarks and on two challenging real-robot tasks that require precise and dexterous bimanual manipulation.
  arXiv  Detail & Related papers  (2026-02-13T07:30:43Z)
• RoboReward: General-Purpose Vision-Language Reward Models for Robotics [124.34685604054312]
  Vision-language models (VLMs) have shown promise as automatic reward models, yet their effectiveness on real robot tasks is poorly understood.<n>We aim to close this gap by introducing RoboReward, a robotics reward dataset and benchmark built on large-scale real-robot corpora.
  arXiv  Detail & Related papers  (2026-01-02T12:47:34Z)
• ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning [32.137320056371784]
  ReinforceGen is a system that combines task decomposition, data generation, imitation learning, and motion planning.<n>It reaches 80% success rate on all tasks with visuomotor controls in the highest reset range setting.<n>Additional ablation studies show that our fine-tuning approaches contribute to an 89% average performance increase.
  arXiv  Detail & Related papers  (2025-12-18T18:32:39Z)
• Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
  We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
  arXiv  Detail & Related papers  (2025-08-13T13:54:51Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.