SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators

• URL: http://arxiv.org/abs/2609.09155v1
• Date: Tue, 08 Sep 2026 17:59:47 GMT
• Title: SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators
• Abstract summary: SyncWorld is an action-conditioned world model that serves as a zero-shot simulator across unseen environments without any additional training.<n>Training with visual calibration contexts teaches the model to interpret actions through visual evidence.<n>Experiments show that SyncWorld can accurately simulate action outcomes in previously unseen settings.
• Score: 61.95179776027961
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models are increasingly used as policy-in-the-loop imagination environments, where reliable rollouts require fine-grained controllability with respect to low-level robot actions. A key obstacle to scaling such models in robotics is that actions are not a universal language in pixel space: changes in visual environment, camera view, robot placement, or embodiment alter how the same numerical action manifests visually, leading to conflicting supervision under mixed training and brittle generalization at deployment. We introduce SyncWorld, an action-conditioned world model that serves as a zero-shot simulator across unseen environments without any additional training. SyncWorld leverages a visual calibration episode---paired frames and actions that showcase all the controllable degrees of freedom---to specify the setup-specific Action--Visual Mapping in context. Training with visual calibration contexts teaches the model to interpret actions through visual evidence and to leverage interaction history when explicit calibration is unavailable. Experiments show that SyncWorld can accurately simulate action outcomes in previously unseen settings, and that its capability of simulating rollouts enables test-time policy improvement without training.

Related papers

• WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models [7.861070986808876]
  Post-training VLA policies typically rely on supervised fine-tuning with expensive and potentially unstable real-world exploration.<n>We introduce WISE (World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models), a unified framework that coordinates when and how world-model imagination is used during policy refinement.
  arXiv  Detail & Related papers  (2026-09-03T11:17:57Z)
• AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization [88.25615187835321]
  We propose AnyWorld, a cross-embodiment world modeling framework.<n>Our model factorizes an interaction into action, camera, and embodiment.<n>We train the model with large-scale human interaction pretraining.
  arXiv  Detail & Related papers  (2026-08-29T12:55:15Z)
• BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generation [22.803470671269874]
  We introduce BehaviorWorldGen, a framework that closes the loop between action models and world simulators.<n>Its core component is BehaviorFlow, a meta-action-conditioned traffic-flow model.<n>The resulting rollouts are rendered by a world simulator into realistic multi-view observations.
  arXiv  Detail & Related papers  (2026-08-23T02:53:09Z)
• DisCo: World Models with Discrete Camera Motion Control [79.86256515640231]
  We propose DisCo, a controllable video world model that conditions generation on a compact set of discrete action primitives to improve action separability.<n>DisCo achieves significantly more reliable action following while preserving visual quality.
  arXiv  Detail & Related papers  (2026-06-06T03:50:45Z)
• Interactive World Simulator for Robot Policy Training and Evaluation [21.481187472784047]
  We present Interactive World Simulator, a framework for building interactive world models from a robot interaction dataset.<n>In our experiments, the learned world models produce interaction-consistent pixel-level predictions.<n>We find that policies trained on world-model-generated data perform comparably to those trained on the same amount of real-world data.
  arXiv  Detail & Related papers  (2026-03-09T16:13:32Z)
• Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
  We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
  arXiv  Detail & Related papers  (2026-01-24T07:43:57Z)
• Pre-Trained Video Generative Models as World Simulators [59.546627730477454]
  We propose Dynamic World Simulation (DWS) to transform pre-trained video generative models into controllable world simulators.<n>To achieve precise alignment between conditioned actions and generated visual changes, we introduce a lightweight, universal action-conditioned module.<n> Experiments demonstrate that DWS can be versatilely applied to both diffusion and autoregressive transformer models.
  arXiv  Detail & Related papers  (2025-02-10T14:49:09Z)
• Learning Interactive Real-World Simulators [96.5991333400566]
  We explore the possibility of learning a universal simulator of real-world interaction through generative modeling. We use the simulator to train both high-level vision-language policies and low-level reinforcement learning policies. Video captioning models can benefit from training with simulated experience, opening up even wider applications.
  arXiv  Detail & Related papers  (2023-10-09T19:42:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.