MBench: A Comprehensive Benchmark on Memory Capability for Video World Models

• URL: http://arxiv.org/abs/2606.00793v2
• Date: Mon, 08 Jun 2026 08:58:38 GMT
• Title: MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
• Abstract summary: We present textbfMBench, a benchmark dedicated to quantifying and evaluating the memory capability of video world models.<n>Our benchmark is built upon rigorously curated real-captured long videos, and evaluated by rule-based quantitative matrices and VLM.
• Score: 36.71271805993198
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in video-based world models have demonstrated an unprecedented ability to synthesize high-fidelity visual sequences. However, a fundamental gap persists between visually plausible video generation and the functional requirements of a world model, particularly in maintaining a stable and reasonable internal state over extended temporal horizons. While existing benchmarks primarily emphasize visual quality, motion coherence, and text-video alignment, they largely overlook memory, the core capability of a world model to preserve consistency across long-term horizons and complex interactions. To address this gap, we present \textbf{MBench}, a comprehensive benchmark dedicated to quantifying and evaluating the memory capability of video world models. We systematically decompose the memory capability of video world models into three hierarchical and complementary core dimensions: entity consistency, environment consistency, and causal consistency, which are further refined into 12 quantifiable sub-dimensions for comprehensive characterization of long-term memory. Our benchmark is built upon rigorously curated real-captured long videos, and evaluated by rule-based quantitative matrices and VLM to enable objective and comprehensive consistency assessment. Extensive evaluations of mainstream state-of-the-art video world models reveal critical systemic limitations of existing methods in long-term state retention, providing a standardized benchmark and clear research direction to advance the field.

Related papers

• WorldOlympiad: Can Your World Model Survive a Triathlon? [39.69359039523777]
  We introduce WorldOlympiad, a benchmark for diagnosing video-based world models across physical faithfulness, geometric consistency, and interaction fidelity.<n>WorldOlympiad decomposes world-model evaluation into three complementary dimensions.
  arXiv  Detail & Related papers  (2026-06-09T17:24:36Z)
• Long-CODE: Isolating Pure Long-Context as an Orthogonal Dimension in Video Evaluation [16.64717198652712]
  We argue that long-video metrics should be disentangled from short-video assessments.<n>We introduce a suite of long-video attribute corruption tests and a novel long-video metric based on shot dynamics.<n>Our proposed metrics achieve state-of-the-art correlation with human judgments.
  arXiv  Detail & Related papers  (2026-04-19T13:17:34Z)
• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory [53.39687409541093]
  Matrix-Game 3.0 is a memory-augmented interactive world model designed for 720p real-time longform video generation.<n>We introduce systematic improvements across data, model, and inference.<n> Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model.
  arXiv  Detail & Related papers  (2026-04-10T06:00:09Z)
• INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling [44.09983529522167]
  INSPATIO-WORLD is a novel real-time framework capable of recovering and generating high-fidelity interactive scenes from a single video.<n>Implicit Spatiotemporal Cache aggregates reference and historical observations into a latent world representation.<n>Explicit Spatial Constraint Module enforces geometric structure and translates user interactions into precise and physically plausible camera trajectories.
  arXiv  Detail & Related papers  (2026-04-08T15:31:22Z)
• Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory [101.2076718776139]
  We propose a robust interactive world model capable of maintaining coherent visual memory over 1000+ frames in complex real-world environments.<n>We introduce a Pose-free Memory (HPMC) that distills historical latents into a fixed-budget geometric representation.<n>We also propose an Uncertainty-aware Action Labeling module that discretizes continuous motion into a tri-state logic.
  arXiv  Detail & Related papers  (2026-02-02T17:52:56Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• LongVie 2: Multimodal Controllable Ultra-Long Video World Model [94.85126937285527]
  LongVie 2 is an end-to-end autoregressive framework trained in three stages.<n>LongVie 2 achieves state-of-the-art performance in long-range controllability, temporal coherence, and visual fidelity.
  arXiv  Detail & Related papers  (2025-12-15T17:59:58Z)
• WorldPack: Compressed Memory Improves Spatial Consistency in Video World Modeling [42.52474988220278]
  We propose WorldPack, a video world model with efficient compressed memory.<n>WorldPack significantly improves spatial consistency, fidelity, and quality in long-term generation.<n>Our performance is evaluated with LoopNav, a benchmark on Minecraft.
  arXiv  Detail & Related papers  (2025-12-02T07:06:23Z)
• Video World Models with Long-term Spatial Memory [110.530715838396]
  We introduce a novel framework to enhance long-term consistency of video world models.<n>Our framework includes mechanisms to store and retrieve information from the long-term spatial memory.<n>Our evaluations show improved quality, consistency, and context length compared to relevant baselines.
  arXiv  Detail & Related papers  (2025-06-05T17:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.