Target-Bench: Can World Models Achieve Mapless Path Planning with Semantic Targets?

• URL: http://arxiv.org/abs/2511.17792v1
• Date: Fri, 21 Nov 2025 21:36:02 GMT
• Title: Target-Bench: Can World Models Achieve Mapless Path Planning with Semantic Targets?
• Abstract summary: Target-Bench is the first benchmark designed to evaluate world models on mapless path planning toward semantic targets in real-world environments.<n>We provide 450 robot-collected video sequences spanning 45 semantic categories with SLAM-based ground truth trajectories.<n>We evaluate state-of-the-art models including Sora 2, Veo 3.1, and the Wan series.
• Score: 25.37180619162052
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While recent world models generate highly realistic videos, their ability to perform robot path planning remains unclear and unquantified. We introduce Target-Bench, the first benchmark specifically designed to evaluate world models on mapless path planning toward semantic targets in real-world environments. Target-Bench provides 450 robot-collected video sequences spanning 45 semantic categories with SLAM-based ground truth trajectories. Our evaluation pipeline recovers camera motion from generated videos and measures planning performance using five complementary metrics that quantify target-reaching capability, trajectory accuracy, and directional consistency. We evaluate state-of-the-art models including Sora 2, Veo 3.1, and the Wan series. The best off-the-shelf model (Wan2.2-Flash) achieves only 0.299 overall score, revealing significant limitations in current world models for robotic planning tasks. We show that fine-tuning an open-source 5B-parameter model on only 325 scenarios from our dataset achieves 0.345 overall score -- an improvement of more than 400% over its base version (0.066) and 15% higher than the best off-the-shelf model. We will open-source the code and dataset.

Related papers

• Pelican-Sim 1.0: A General World Model Simulator for Embodied Intelligence [22.0636262013758]
  Pelican-Sim 1.0 is a general world model simulator for embodied intelligence.<n>It predicts future observations from visual context and robot actions to support downstream learning and decision making.
  arXiv  Detail & Related papers  (2026-09-10T17:51:01Z)
• RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction [24.674633972725804]
  We introduce RoboPhys-3D, a 3D-grounded embodied world model (EWM) benchmark built on RoboTwin 2.0.<n>A defining feature of RoboPhys-3D is that generated and ground-truth videos are processed through the same 3D reconstruction pipeline.<n>The benchmark organizes 50 complementary metrics into 18 sub-dimensions across four levels: pixel-level fidelity, 3D geometry consistency, state-level understanding, and task-level completeness.
  arXiv  Detail & Related papers  (2026-08-28T09:44:13Z)
• GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation [73.02743604295246]
  WMBench is a benchmark constructed from real-robot teleoperation data and matched policy rollouts.<n>We analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions.<n>Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism.
  arXiv  Detail & Related papers  (2026-07-02T17:02:43Z)
• IMWM: Intuition Models Complement World Models for Latent Planning [21.033541866844683]
  We show that a finite-budget sample-based planner still fails on some tasks, indicating that the bottleneck can lie in search rather than in world-model accuracy.<n>Motivated by this gap, we propose IMWM, which pairs the world model with an intuition model trained from demonstrations to recognize promising actions.
  arXiv  Detail & Related papers  (2026-06-01T03:23:25Z)
• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory [53.39687409541093]
  Matrix-Game 3.0 is a memory-augmented interactive world model designed for 720p real-time longform video generation.<n>We introduce systematic improvements across data, model, and inference.<n> Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model.
  arXiv  Detail & Related papers  (2026-04-10T06:00:09Z)
• HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents [71.09493646932046]
  We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents.<n>The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning.<n>Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro.
  arXiv  Detail & Related papers  (2026-04-08T17:59:48Z)
• Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling [51.40150411616207]
  We introduce Latent Particle World Model (LPWM), a self-supervised object-centric world model scaled to real-world multi-object datasets.<n>LPWM autonomously discovers keypoints, bounding boxes, and object masks directly from video data.<n>Our architecture is trained end-to-end purely from videos and supports flexible conditioning on actions, language, and image goals.
  arXiv  Detail & Related papers  (2026-03-04T19:36:08Z)
• Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
  We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
  arXiv  Detail & Related papers  (2026-01-24T07:43:57Z)
• Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test [62.17144846428715]
  We introduce the Embodied Turing Test benchmark: WoW-World-Eval (Wow,wo,val)<n>Wow-wo-val examines five core abilities, including perception, planning, prediction, generalization and execution.<n>For the Inverse Dynamic Model Turing Test, we first use an IDM to evaluate the video foundation models' execution accuracy in the real world.
  arXiv  Detail & Related papers  (2026-01-07T17:50:37Z)
• Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report [13.66563834458748]
  1X World Model Challenge introduces an open-source benchmark of real-world humanoid interaction.<n>We adapt the video generation foundation model Wan-2.2 TI2V-5B to video-state-conditioned future frame prediction.<n>Our models achieve 23.0 dB PSNR in the sampling task and a Top-500 CE of 6.6386 in the compression task, securing 1st place in both challenges.
  arXiv  Detail & Related papers  (2025-10-08T14:49:12Z)
• WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning [52.36434784963598]
  We introduce WorldPrediction, a video-based benchmark for evaluating world modeling and procedural planning capabilities of different AI models.<n>We show that current frontier models barely achieve 57% accuracy on WorldPrediction-WM and 38% on WorldPrediction-PP whereas humans are able to solve both tasks perfectly.
  arXiv  Detail & Related papers  (2025-06-04T18:22:40Z)
• BOP Challenge 2024 on Model-Based and Model-Free 6D Object Pose Estimation [55.13521733366838]
  The 6th in a series of public competitions organized to capture the state of the art in 6D object pose estimation and related tasks.<n>In 2024, we introduced new model-free tasks, where no 3D object models are available and methods need to onboard objects just from provided reference videos.<n>We defined a new, more practical 6D object detection task where identities of objects visible in a test image are not provided as input.
  arXiv  Detail & Related papers  (2025-04-03T17:55:19Z)
• WorldModelBench: Judging Video Generation Models As World Models [57.776769550453594]
  Video generation models have rapidly progressed, positioning themselves as video world models capable of supporting decision-making applications like robotics and autonomous driving.<n>Current benchmarks fail to rigorously evaluate these claims, focusing only on general video quality.<n>We propose WorldModelBench, a benchmark designed to evaluate the world modeling capabilities of video generation models in application-driven domains.
  arXiv  Detail & Related papers  (2025-02-28T03:58:23Z)
• Beyond One-Size-Fits-All: Tailored Benchmarks for Efficient Evaluation [19.673388630963807]
  We present TailoredBench, a method that conducts customized evaluation tailored to each target model.<n>A Global-coreset is first constructed as a probe to identify the most consistent source models for each target model.<n>A scalable K-Medoids clustering algorithm is proposed to extend the Global-coreset to a tailored Native-coreset for each target model.
  arXiv  Detail & Related papers  (2025-02-19T09:31:50Z)
• A Control-Centric Benchmark for Video Prediction [69.22614362800692]
  We propose a benchmark for action-conditioned video prediction in the form of a control benchmark. Our benchmark includes simulated environments with 11 task categories and 310 task instance definitions. We then leverage our benchmark to study the effects of scaling model size, quantity of training data, and model ensembling.
  arXiv  Detail & Related papers  (2023-04-26T17:59:45Z)
• Unifying Language Learning Paradigms [96.35981503087567]
  We present a unified framework for pre-training models that are universally effective across datasets and setups. We show how different pre-training objectives can be cast as one another and how interpolating between different objectives can be effective. Our model also achieve strong results at in-context learning, outperforming 175B GPT-3 on zero-shot SuperGLUE and tripling the performance of T5-XXL on one-shot summarization.
  arXiv  Detail & Related papers  (2022-05-10T19:32:20Z)
• Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin-picking [98.5984733963713]
  We propose an iterative self-training framework for sim-to-real 6D object pose estimation to facilitate cost-effective robotic grasping. We establish a photo-realistic simulator to synthesize abundant virtual data, and use this to train an initial pose estimation network. This network then takes the role of a teacher model, which generates pose predictions for unlabeled real data.
  arXiv  Detail & Related papers  (2022-04-14T15:54:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.