MIND: Benchmarking Memory Consistency and Action Control in World Models

• URL: http://arxiv.org/abs/2602.08025v2
• Date: Wed, 11 Feb 2026 18:42:39 GMT
• Title: MIND: Benchmarking Memory Consistency and Action Control in World Models
• Abstract summary: We introduce MIND, the first open-domain closed-loop revisited benchmark for evaluating Memory consIstency and action coNtrol in worlD models.<n>MIND contains 250 high-quality videos at 1080p and 24 FPS, including 100 (first-person) + 100 (third-person) video clips under a shared action space.<n>We design various action spaces, including different character movement speeds and camera rotation angles, to evaluate the action generalization capability across different action spaces under shared scenes.
• Score: 28.346879515303755
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: World models aim to understand, remember, and predict dynamic visual environments, yet a unified benchmark for evaluating their fundamental abilities remains lacking. To address this gap, we introduce MIND, the first open-domain closed-loop revisited benchmark for evaluating Memory consIstency and action coNtrol in worlD models. MIND contains 250 high-quality videos at 1080p and 24 FPS, including 100 (first-person) + 100 (third-person) video clips under a shared action space and 25 + 25 clips across varied action spaces covering eight diverse scenes. We design an efficient evaluation framework to measure two core abilities: memory consistency and action control, capturing temporal stability and contextual coherence across viewpoints. Furthermore, we design various action spaces, including different character movement speeds and camera rotation angles, to evaluate the action generalization capability across different action spaces under shared scenes. To facilitate future performance benchmarking on MIND, we introduce MIND-World, a novel interactive Video-to-World baseline. Extensive experiments demonstrate the completeness of MIND and reveal key challenges in current world models, including the difficulty of maintaining long-term memory consistency and generalizing across action spaces. Code: https://github.com/CSU-JPG/MIND.

Related papers

• WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models [37.59051701511746]
  We introduce WorldRoamBench, an open-world benchmark for long-horizon stability across four dimensions.<n>The benchmark comprises 600+ test cases across Nature, Urban, and Indoor scenes in first/third-person views with WASD 10-60s continuous interaction.
  arXiv  Detail & Related papers  (2026-06-30T13:51:44Z)
• Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulation [55.42006264038458]
  Action-conditioned world models have emerged as a promising paradigm for robot learning.<n>Mem-World is a memory-augmented multi-view action-conditioned world model.<n>We present W-VMem, a 4D wrist-view-centered surfel-indexed memory that anchors historical observations to temporally evolving surface elements.
  arXiv  Detail & Related papers  (2026-06-17T11:42:00Z)
• DisCo: World Models with Discrete Camera Motion Control [79.86256515640231]
  We propose DisCo, a controllable video world model that conditions generation on a compact set of discrete action primitives to improve action separability.<n>DisCo achieves significantly more reliable action following while preserving visual quality.
  arXiv  Detail & Related papers  (2026-06-06T03:50:45Z)
• WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation [56.5415838759151]
  WBench is a comprehensive benchmark for evaluation of interactive world models.<n>It covers video quality, setting adherence, interaction adherence, consistency, and physics compliance.<n>It contains 289 test cases and 1,058 interaction turns, covering diverse scenes, styles, subjects, and both first- and third-person perspectives.
  arXiv  Detail & Related papers  (2026-05-25T14:01:31Z)
• LARY: A Latent Action Representation Yielding Benchmark for Generalizable Vision-to-Action Alignment [19.95295518800639]
  General visual foundation models consistently outperform specialized embodied latent action models.<n>Latent-based visual space is better aligned to physical action space than pixel-based space.
  arXiv  Detail & Related papers  (2026-04-13T16:30:35Z)
• ActionParty: Multi-Subject Action Binding in Generative Video Games [117.52562594944679]
  ActionParty is a controllable multi-subject world model for generative video games.<n>We demonstrate the first video world model capable of controlling up to seven players simultaneously across 46 diverse environments.
  arXiv  Detail & Related papers  (2026-04-02T17:59:58Z)
• Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models [56.44348799741838]
  We introduce Hybrid Memory, a novel paradigm requiring models to simultaneously act as precise archivists for static backgrounds and vigilant trackers for dynamic subjects.<n>We construct HM-World, the first large-scale video dataset dedicated to hybrid memory.<n>We also propose HyDRA, a specialized memory architecture that compresses memory into tokens and utilizes atemporal relevance-driven retrieval mechanism.
  arXiv  Detail & Related papers  (2026-03-26T17:56:01Z)
• Olaf-World: Orienting Latent Actions for Video World Modeling [100.96069208914957]
  Scaling action-controllable world models is limited by the scarcity of action labels.<n>We present Olaf-World, a pipeline that pretrains action-conditioned video world models from large-scale passive video.
  arXiv  Detail & Related papers  (2026-02-10T18:58:41Z)
• DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos [110.98100817695307]
  We introduce DreamDojo, a foundation world model that learns diverse interactions and dexterous controls from 44k hours of egocentric human videos.<n>Our work enables several important applications based on generative world models, including live teleoperation, policy evaluation, and model-based planning.
  arXiv  Detail & Related papers  (2026-02-06T18:49:43Z)
• DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learning [24.808926786222376]
  We present DreamActor-M2, a universal animation framework that reimagines motion conditioning as an in-context learning problem.<n>Our approach follows a two-stage paradigm. First, we bridge the input modality gap by fusing reference appearance and motion cues into a unified latent space.<n>Second, we introduce a self-bootstrapped data synthesis pipeline that curates pseudo cross-identity training pairs.
  arXiv  Detail & Related papers  (2026-01-29T13:43:17Z)
• Towards High-Consistency Embodied World Model with Multi-View Trajectory Videos [24.111891848073288]
  Embodied world models aim to predict and interact with the physical world through visual observations and actions.<n>MTV-World introduces Multi-view Trajectory-Video control for precise visuomotor prediction.<n>MTV-World achieves precise control execution and accurate physical interaction modeling in complex dual-arm scenarios.
  arXiv  Detail & Related papers  (2025-11-17T02:17:04Z)
• SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding [48.64661382961745]
  We introduce Spatio-temporal Video Action Grounding (SVAG), a novel task that requires models to simultaneously detect, track, and temporally localize all referent objects in videos.<n>To support this task, we construct SVAG-Bench, a large-scale benchmark comprising 688 videos, 19,590 annotated records, and 903 unique verbs.<n> Empirical results show that existing models perform poorly on SVAG, particularly in dense or complex scenes.
  arXiv  Detail & Related papers  (2025-10-14T22:10:49Z)
• IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video? [56.33950760097989]
  IWR-Bench is a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video.<n>IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions.<n>This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code.
  arXiv  Detail & Related papers  (2025-09-29T12:38:06Z)
• HumanVideo-MME: Benchmarking MLLMs for Human-Centric Video Understanding [120.84817886550765]
  Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks involving both images and videos.<n>Existing human-centric benchmarks predominantly emphasize video generation quality and action recognition, while overlooking essential perceptual and cognitive abilities required in human-centered scenarios.<n>We propose a rigorously curated benchmark designed to provide a more holistic evaluation of MLLMs in human-centric video understanding.
  arXiv  Detail & Related papers  (2025-07-07T11:52:24Z)
• VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory [55.73900731190389]
  We introduce Surfel-Indexed View Memory (VMem), a memory module that remembers past views by indexing them geometrically based on the 3D surface elements (surfels) they have observed.<n>VMem enables efficient retrieval of the most relevant past views when generating new ones.
  arXiv  Detail & Related papers  (2025-06-23T17:59:56Z)
• Training-Free Robust Interactive Video Object Segmentation [82.05906654403684]
  We propose a training-free prompt tracking framework for interactive video object segmentation (I-PT) We jointly adopt sparse points and boxes tracking, filtering out unstable points and capturing object-wise information. Our framework has demonstrated robust zero-shot video segmentation results on popular VOS datasets.
  arXiv  Detail & Related papers  (2024-06-08T14:25:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.