Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision

• URL: http://arxiv.org/abs/2605.28865v1
• Date: Fri, 22 May 2026 03:31:47 GMT
• Title: Emergent Semantic Representations in World Models through Physical Interaction without Linguistic Supervision
• Abstract summary: We train a VAE-based world model on random embodied exploration.<n>We find that its latent space develops spatial structure semantic that mirrors physical geometry.<n>These findings establish physical world geometry as the organizing principle of world model representations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: What does a world model learn from physical exploration, without any linguistic supervision? We argue the answer is organized by a single principle: the geometric structure of the physical world. Training a VAE-based world model on random embodied exploration, we find that its latent space develops spatial semantic structure that mirrors physical geometry -- direction accuracy 0.677+-0.029 versus 0.547 for a randomly initialized encoder, and position RSA 0.192+-0.047 versus 0.029 for random encoders (6.6x improvement), showing that training induces genuine structural organization beyond CNN inductive bias. Across 20 temporal checkpoints, prediction performance and semantic alignment co-improve (Spearman r=-0.61, p=0.004), consistent with the shared-driver account. We confirm this through a double knockout: standard KL regularization (beta=0.1) forces the encoder away from geometric structure, and both prediction performance and semantic alignment collapse simultaneously to near-chance by step 50,000 -- exactly as the shared-driver account predicts. Reducing beta to 0.001 restores geometric access and recovers both capabilities together. These findings establish physical world geometry as the organizing principle of world model representations, with direct implications for the design of semantically grounded embodied agents.

Related papers

• PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models [60.567497085815724]
  PhysBrain 1.5 is a unified model for understanding physical environments, generating actions, and predicting future states.<n>Pre-training draws its embodied supervision entirely from human interaction videos.<n>It achieves the best open-source results on 14 benchmarks while retaining general multimodal capabilities.
  arXiv  Detail & Related papers  (2026-09-14T03:25:46Z)
• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representations [18.546826658944806]
  POKEWORLD is an interactive environment whose visually identical objects hide mass, drag, and contact stiffness.<n>A certificate-gated protocol first certifies each parameter as recoverable from raw observations, then measures whether it enters the latent.<n>On RH20T, an input-target factorial across scaling curves reproduces both mechanisms across two robots and 4,258 episodes.
  arXiv  Detail & Related papers  (2026-07-29T15:15:30Z)
• FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry [52.232826445242644]
  FoundationGeo bridges relative and metric prediction via spatial calibration and principled data design.<n>Stage 1 learns a high-fidelity, affine-invariant geometry model by initializing with DINOv3.<n>Stage 2 moves beyond global scaling by introducing pixel-wise calibration fields for metric estimation.
  arXiv  Detail & Related papers  (2026-07-13T14:10:01Z)
• ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI [110.18670135256235]
  Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, monitor progress, and improve from experience.<n>We present ACE-Brain-0.5, a unified embodied foundation model that organizes robot intelligence into five coupled functions.<n>A single 8B backbone instantiates the first four functions: grounding objects and affordances, reasoning over 3D and egocentric spatial relations, decomposing instructions into subgoals, generating navigation and manipulation actions, and estimating progress for verification and recovery.
  arXiv  Detail & Related papers  (2026-07-05T17:43:06Z)
• Is Your Driving World Model an All-Around Player? [107.28582946012818]
  We introduce WorldLens, a unified benchmark that measures world-model fidelity across the full spectrum.<n>Our evaluation of six representative models reveals that no existing approach dominates across all axes.<n>To bridge algorithmic metrics with human perception, we contribute WorldLens-26K, a 26,808-entry human-annotated preference dataset.
  arXiv  Detail & Related papers  (2026-05-11T17:05:49Z)
• HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation [83.31948299340782]
  HERMES++ is a unified driving world model that integrates 3D scene understanding and future geometry prediction within a single framework.<n>Hermes++ achieves strong performance, outperforming specialist approaches in both future point cloud prediction and 3D scene understanding tasks.
  arXiv  Detail & Related papers  (2026-04-30T17:59:58Z)
• ATLAS: Constitution-Conditioned Latent Geometry and Redistribution Across Language Models and Neural Perturbation Data [0.0]
  Constitution-conditioned post-training can be analysed as a structured perturbation of a model's learned representational geometry.<n>We introduce ATLAS, a geometry-first program that traces constitution-induced hidden-state structure across charts, models, and substrates.
  arXiv  Detail & Related papers  (2026-04-19T23:26:02Z)
• AI Mental Models: Learned Intuition and Deliberation in a Bounded Neural Architecture [0.0]
  This paper asks whether a bounded neural architecture can exhibit a meaningful division of labor between intuition and deliberation on a syllogistic reasoning benchmark.<n>Experiment 1 evaluates a direct neural baseline for predicting full 9-way human response distributions under 5-fold cross-validation.<n>Experiment 2 introduces a bounded dual-path architecture with separate intuition and deliberation pathways, motivated by computational mental-model theory.
  arXiv  Detail & Related papers  (2026-03-23T20:44:38Z)
• Thinking in Different Spaces: Domain-Specific Latent Geometry Survives Cross-Architecture Translation [0.7162422068114824]
  We investigate whether independently trained language models converge to geometrically compatible latent representations.<n>We learn a linear projection matrix that maps activation vectors from a large teacher model into the coordinate system of a smaller student model.<n>We report a near-zero correlation between geometric alignment quality and behavioral correction rate.
  arXiv  Detail & Related papers  (2026-03-20T18:26:23Z)
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels [49.35636088613484]
  Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces.<n>We introduce LeModelWorld, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms.<n>With 15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models.
  arXiv  Detail & Related papers  (2026-03-13T19:48:14Z)
• Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement [0.0]
  Vision-language models encode geometry that their text pathway fails to express.<n>LoRA fine-tuning (r=16, 2,000 images) narrows this gap to 6.5 degrees.<n>These findings enable a single frozen backbone to function as a multi-task geometric sensor.
  arXiv  Detail & Related papers  (2026-03-06T16:48:27Z)
• Geometric Priors for Generalizable World Models via Vector Symbolic Architecture [9.216794073296679]
  Key challenge in artificial intelligence is understanding how neural systems learn representations that capture the underlying dynamics of the world.<n>We address these issues with a generalizable world model grounded in Vector Symbolic Architecture (VSA) principles as geometric priors.<n>We show how training to have latent group structure yields generalizable, data-efficient, and interpretable world models.
  arXiv  Detail & Related papers  (2026-02-25T00:41:42Z)
• WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World [100.68103378427567]
  Generative world models are reshaping embodied AI, enabling agents to synthesize realistic 4D driving environments that look convincing but often fail physically or behaviorally.<n>We introduce WorldLens, a full-spectrum benchmark evaluating how well a model builds, understands, and behaves within its generated world.<n>We further construct WorldLens-26K, a large-scale dataset of human-annotated videos with numerical scores and textual rationales, and develop WorldLens-Agent.
  arXiv  Detail & Related papers  (2025-12-11T18:59:58Z)
• Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence [80.6840060272386]
  This paper identifies the importance of being geometry-aware for semantic correspondence. We show that incorporating this information can markedly enhance semantic correspondence performance. Our method achieves a PCK@0.10 score of 65.4 (zero-shot) and 85.6 (supervised) on the challenging SPair-71k dataset.
  arXiv  Detail & Related papers  (2023-11-28T18:45:13Z)
• Unifying Language Learning Paradigms [96.35981503087567]
  We present a unified framework for pre-training models that are universally effective across datasets and setups. We show how different pre-training objectives can be cast as one another and how interpolating between different objectives can be effective. Our model also achieve strong results at in-context learning, outperforming 175B GPT-3 on zero-shot SuperGLUE and tripling the performance of T5-XXL on one-shot summarization.
  arXiv  Detail & Related papers  (2022-05-10T19:32:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.