Related papers: PhysBrain 1.0 Technical Report

PhysBrain 1.0 Technical Report

URL: http://arxiv.org/abs/2605.15298v1
Date: Thu, 14 May 2026 18:11:47 GMT
Title: PhysBrain 1.0 Technical Report
Abstract summary: PhysBrain 1.0 converts large-scale human egocentric video into structured physical commonsense supervision before robot adaptation.<n>Our data engine extracts scene elements, spatial dynamics, action execution, and depth-aware relations.<n>Results suggest that scaling physical commonsense from human interaction video can provide an effective bridge from multimodal understanding to robot action.
Score: 31.13440306363037
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Vision-language-action models have advanced rapidly, but robot trajectories alone provide limited coverage for learning broad physical understanding. PhysBrain 1.0 studies a complementary route: converting large-scale human egocentric video into structured physical commonsense supervision before robot adaptation. Our data engine extracts scene elements, spatial dynamics, action execution, and depth-aware relations, then turns them into question-answer supervision for training PhysBrain VLMs. The resulting physical priors are further transferred to VLA policies through a capability-preserving and language-sensitive adaptation design. Across multimodal QA benchmarks and embodied control benchmarks, including ERQA, PhysBench, SimplerEnv-WidowX, LIBERO, and RoboCasa, PhysBrain 1.0 achieves SOTA results and shows especially strong out-of-domain performance on SimplerEnv. These results suggest that scaling physical commonsense from human interaction video can provide an effective bridge from multimodal understanding to robot action.

Related papers

PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models [60.567497085815724]
PhysBrain 1.5 is a unified model for understanding physical environments, generating actions, and predicting future states.<n>Pre-training draws its embodied supervision entirely from human interaction videos.<n>It achieves the best open-source results on 14 benchmarks while retaining general multimodal capabilities.
arXiv Detail & Related papers (2026-09-14T03:25:46Z)
Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation [63.04810454548667]
We introduce PhysGen, a scalable continuous and sequential world interaction framework to solve robotic manipulation tasks.<n>By treating the pretrained video model as a proxy for a physics simulator, PhysGen models the dynamic interplay between the external environment and robot actions.<n>We introduce a multimodal continuous representation that unifies video and action into shared physical tokens, bridging the gap between discrete video generation and continuous robotic control.
arXiv Detail & Related papers (2026-02-18T14:58:18Z)
Mind to Hand: Purposeful Robotic Control via Embodied Reasoning [12.275897522668858]
We introduce Lumo-1, a model that unifies robot reasoning ("mind") with robot action ("hand")<n>Our approach builds upon the general multi-modal reasoning capabilities of pre-trained vision-language models (VLMs)<n>We integrate reinforcement learning to further refine reasoning-action consistency and close the loop between semantic inference and motor control.
arXiv Detail & Related papers (2025-12-09T13:19:37Z)
Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
arXiv Detail & Related papers (2025-08-13T13:54:51Z)
PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability [31.532470258146073]
We propose a unified representation of physical reachability across diverse robots, i.e., Space-Physical Reachability Map (S-P Map)<n> PhysVLM is a vision-language model that integrates this reachability information into visual reasoning.
arXiv Detail & Related papers (2025-03-11T14:34:41Z)
Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulation [90.00687889213991]
Solving complex long-horizon robotic manipulation problems requires sophisticated high-level planning capabilities.<n>Vision-language models (VLMs) pretrained on Internet data could in principle offer a framework for tackling such problems.<n>In this paper, we introduce a novel test-time framework that enhancesVLMs' physical reasoning capabilities for multi-stage manipulation tasks.
arXiv Detail & Related papers (2025-02-23T20:42:15Z)
Physically Grounded Vision-Language Models for Robotic Manipulation [59.143640049407104]
We propose PhysObjects, an object-centric dataset of 39.6K crowd-sourced and 417K automated physical concept annotations. We show that fine-tuning a vision-language model on PhysObjects improves its understanding of physical object concepts. We incorporate this physically grounded VLM in an interactive framework with a large language model-based robotic planner.
arXiv Detail & Related papers (2023-09-05T20:21:03Z)
Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language [92.7638697243969]
We propose a unified framework that can jointly learn visual concepts and infer physics models of objects from videos and language. This is achieved by seamlessly integrating three components: a visual perception module, a concept learner, and a differentiable physics engine.
arXiv Detail & Related papers (2021-10-28T17:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.