WorldString: Actionable World Representation

• URL: http://arxiv.org/abs/2605.18743v2
• Date: Wed, 20 May 2026 07:13:29 GMT
• Title: WorldString: Actionable World Representation
• Abstract summary: We propose WorldString, a neural architecture capable of modeling the state manifold of real-world objects by learning directly from point clouds or RGB-D video streams.<n>WorldString acts as a foundational building block for physical world models; thus, we name it WorldString. Sweetly, its fully differentiable structure seamlessly enables future integration with policy learning and neural dynamics.
• Score: 29.267308971639682
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inspired by the emergent behaviors in large language models that generalized human intelligence, the research community is pursuing similar emergent capabilities within world models, with a emphasis on modeling the physical world. Within the scope of physical world model, objects are the fundamental primitives that constitute physical reality. From humans to computers, nearly everything we interact with is an object. These objects are rarely static; they are actionable entities with varying states determined by their intrinsic properties. While current methods approach object action states either via video generation or dynamic scene reconstruction, none explicitly model this basic element in a unified, principled way to build an actionable object representation. We propose WorldString, a neural architecture capable of modeling the state manifold of real-world objects by learning directly from point clouds or RGB-D video streams. Serving as a versatile digital twin, it acts as a foundational building block for physical world models; thus, we name it WorldString. Sweetly, its fully differentiable structure seamlessly enables future integration with policy learning and neural dynamics.

Related papers

• Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning [79.29811222102721]
  We introduce Code-as-World, a paradigm that represents physical worlds through executable world representations.<n>By expressing physical composition, dynamic evolution, and visual appearance as executable code, Code-as-World provides a compact, quantitatively grounded, and controllable abstraction of the physical world.<n>We use verified executable worlds to provide scalable physical supervision for training vision-language models on quantitative physical reasoning.
  arXiv  Detail & Related papers  (2026-08-27T17:57:25Z)
• PhiZero: A World Model Built Around Physical Language [50.268271884186845]
  PhiZero is a physical world model built around physical language.<n>It infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos.<n>We show its potential for realistic and interactive world modeling, fine-grained action-conditioned simulation, and zero-shot motion transfer.
  arXiv  Detail & Related papers  (2026-07-30T17:59:46Z)
• Physical Object Understanding with a Physically Controllable World Model [11.085294403458972]
  A central challenge in visual intelligence is learning the physical structure of scenes from raw videos.<n>We introduce a new class of probabilistic world models that support estimation of the probability of any visual variable.<n>We show that our world model can manipulate objects in 3D, enabling applications such as Visual Jenga.
  arXiv  Detail & Related papers  (2026-05-30T00:10:32Z)
• Is Your Driving World Model an All-Around Player? [107.28582946012818]
  We introduce WorldLens, a unified benchmark that measures world-model fidelity across the full spectrum.<n>Our evaluation of six representative models reveals that no existing approach dominates across all axes.<n>To bridge algorithmic metrics with human perception, we contribute WorldLens-26K, a 26,808-entry human-annotated preference dataset.
  arXiv  Detail & Related papers  (2026-05-11T17:05:49Z)
• WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World [100.68103378427567]
  Generative world models are reshaping embodied AI, enabling agents to synthesize realistic 4D driving environments that look convincing but often fail physically or behaviorally.<n>We introduce WorldLens, a full-spectrum benchmark evaluating how well a model builds, understands, and behaves within its generated world.<n>We further construct WorldLens-26K, a large-scale dataset of human-annotated videos with numerical scores and textual rationales, and develop WorldLens-Agent.
  arXiv  Detail & Related papers  (2025-12-11T18:59:58Z)
• Simulating the Visual World with Artificial Intelligence: A Roadmap [48.64639618440864]
  Video generation is shifting from generating visually appealing clips to building virtual environments that support interaction and maintain physical plausibility.<n>This survey provides a systematic overview of this evolution, conceptualizing modern video foundation models as the combination of two core components.<n>We trace the progression of video generation through four generations, culminating in a video generation model that embodies intrinsic physical plausibility.
  arXiv  Detail & Related papers  (2025-11-11T18:59:50Z)
• PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis [52.905353023326306]
  We propose PhysWorld, a framework that synthesizes physically plausible and diverse demonstrations to learn efficient world models.<n>Experiments show that PhysWorld has competitive performance while enabling inference speeds 47 times faster than the recent state-of-the-art method, i.e., PhysTwin.
  arXiv  Detail & Related papers  (2025-10-24T13:25:39Z)
• Dyn-O: Building Structured World Models with Object-Centric Representations [42.65409148846005]
  We introduce Dyn-O, an enhanced structured world model built upon object-centric representations.<n>Compared to prior work in object-centric representations, Dyn-O improves in both learning representations and modeling dynamics.<n>We find that our method can learn object-centric world models directly from pixel observations, outperforming DreamerV3 in rollout prediction accuracy.
  arXiv  Detail & Related papers  (2025-07-04T05:06:15Z)
• Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination [25.62602420895531]
  DreMa is a new approach for constructing digital twins using learned explicit representations of the real world and its dynamics.<n>We show that DreMa can successfully learn novel physical tasks from just a single example per task variation.
  arXiv  Detail & Related papers  (2024-12-19T15:38:15Z)
• Hierarchical Relational Inference [80.00374471991246]
  We propose a novel approach to physical reasoning that models objects as hierarchies of parts that may locally behave separately, but also act more globally as a single whole. Unlike prior approaches, our method learns in an unsupervised fashion directly from raw visual images. It explicitly distinguishes multiple levels of abstraction and improves over a strong baseline at modeling synthetic and real-world videos.
  arXiv  Detail & Related papers  (2020-10-07T20:19:10Z)
• RELATE: Physically Plausible Multi-Object Scene Synthesis Using Structured Latent Spaces [77.07767833443256]
  We present RELATE, a model that learns to generate physically plausible scenes and videos of multiple interacting objects. In contrast to state-of-the-art methods in object-centric generative modeling, RELATE also extends naturally to dynamic scenes and generates videos of high visual fidelity.
  arXiv  Detail & Related papers  (2020-07-02T17:27:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.