PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models

• URL: http://arxiv.org/abs/2609.14973v1
• Date: Mon, 14 Sep 2026 03:25:46 GMT
• Title: PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models
• Abstract summary: PhysBrain 1.5 is a unified model for understanding physical environments, generating actions, and predicting future states.<n>Pre-training draws its embodied supervision entirely from human interaction videos.<n>It achieves the best open-source results on 14 benchmarks while retaining general multimodal capabilities.
• Score: 60.567497085815724
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present PhysBrain 1.5, a unified model for understanding physical environments, generating actions, and predicting future states. Motivated by the physical loop of observation, interaction, and environmental change, we bring these capabilities into a common learning framework. Starting from a general vision--language model, we encode language responses, end-effector motion, and dense visual targets as discrete sequences and jointly optimize them with autoregressive next-token prediction. Pre-training draws its embodied supervision entirely from human interaction videos, using task-centered episodes to pair semantic and spatial context with recovered motion and subsequent observations. We then adapt the model through supervised fine-tuning on a mixture of human demonstrations, robot trajectories, and simulated experience. Across 28 embodied understanding benchmarks, our 8B model achieves an average score of 72.5, setting a new open-source state of the art and performing on par with leading proprietary models such as GPT-6-Astra and Gemini 3.6 Flash. It achieves the best open-source results on 14 benchmarks while retaining general multimodal capabilities. Beyond these understanding evaluations, qualitative examples show the model's ability to produce end-effector trajectories and predict future scenes through spatially aligned RGB, depth, and robot-mask outputs.

Related papers

• AFUN: Towards an Affordance Foundation Model for Functionality Understanding [12.890216832485647]
  We present ourmodel, a step towards an affordance foundation model for functionality understanding.<n>We build a large-scale standardized data pipeline that converts heterogeneous robot, human, simulation, and real-world scan data into a shared affordance schema.<n>Ourmodel outperforms all baselines by a large margin across 8 test sets from 4 benchmarks.
  arXiv  Detail & Related papers  (2026-06-01T17:50:16Z)
• GEM: Generative Supervision Helps Embodied Intelligence [60.28190143133428]
  Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics.<n>A significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in embodied environments.<n>We introduce GEM, a Generative-supervised Embodied vision-language Model designed to bridge this divide.
  arXiv  Detail & Related papers  (2026-05-27T14:39:42Z)
• Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretraining [28.30092786035367]
  DeFI is a novel framework that Decouples visual Forward and Inverse dynamics pretraining to exploit respective data sources.<n>We introduce the General Forward Dynamics Model (GFDM), pretrained on diverse human and robot videos for future prediction, and the General Inverse Dynamics Model (GIDM), trained via self-supervised learning to infer latent actions from unlabeled video transitions.<n>Experiments on CALVIN ABC-D and SimplerEnv demonstrate state-of-the-art performance, with DeFI achieving an average task length of 4.51 for CALVIN, 51.2% success rate on SimplerEnv-Frac
  arXiv  Detail & Related papers  (2026-03-27T17:20:10Z)
• Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation [63.04810454548667]
  We introduce PhysGen, a scalable continuous and sequential world interaction framework to solve robotic manipulation tasks.<n>By treating the pretrained video model as a proxy for a physics simulator, PhysGen models the dynamic interplay between the external environment and robot actions.<n>We introduce a multimodal continuous representation that unifies video and action into shared physical tokens, bridging the gap between discrete video generation and continuous robotic control.
  arXiv  Detail & Related papers  (2026-02-18T14:58:18Z)
• InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation [77.07565723756119]
  We present InternVLA-A1, a vision-language-action model with dynamic predictive capabilities.<n>We pre-train these models on heterogeneous data sources over real-world robot data, synthetic simulation data, and human videos.<n>We evaluate InternVLA-A1 on 12 real-world robotic tasks and a simulation benchmark.
  arXiv  Detail & Related papers  (2026-01-05T18:54:29Z)
• MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Models [45.450035386882824]
  Vision Language Models (VLMs) perform well on standard video tasks but struggle with physics-driven reasoning involving motion dynamics and spatial interactions.<n>We present an approach that addresses this gap by translating physical-world context cues into interpretable representations aligned with VLMs' perception, comprehension, and reasoning.
  arXiv  Detail & Related papers  (2025-11-23T09:43:44Z)
• Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation [54.3628937181904]
  Internal world models (WMs) enable agents to understand the world's state and predict transitions.<n>Recent large Vision-Language Models (VLMs), such as OpenAI o3, GPT-4o and Gemini, exhibit potential as general-purpose WMs.
  arXiv  Detail & Related papers  (2025-06-27T03:24:29Z)
• Human locomotor control timescales depend on the environmental context and sensory input modality [37.48294298569551]
  We present a unified data-driven framework to quantify the control timescales.<n>We apply this framework across tasks including walking and running.<n>Our framework reveals the factors that influence locomotor foot placement control timescales.
  arXiv  Detail & Related papers  (2025-03-20T16:57:15Z)
• λ: A Benchmark for Data-Efficiency in Long-Horizon Indoor Mobile Manipulation Robotics [11.901933884058021]
  We introduce the LAMBDA benchmark-Long-horizon Actions for Mobile-manipulation Benchmarking of Directed Activities.<n>Our benchmark includes 571 human-collected demonstrations that provide realism and diversity in simulated and real-world settings.<n>We find that learning methods, even when pretrained, yield lower success rates, while a neuro-symbolic method performs significantly better and requires less data.
  arXiv  Detail & Related papers  (2024-11-28T19:31:50Z)
• Masked World Models for Visual Control [90.13638482124567]
  We introduce a visual model-based RL framework that decouples visual representation learning and dynamics learning. We demonstrate that our approach achieves state-of-the-art performance on a variety of visual robotic tasks.
  arXiv  Detail & Related papers  (2022-06-28T18:42:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.