Exploring Physical Intelligence Emergence via Omni-Modal Architecture and Physical Data Engine

• URL: http://arxiv.org/abs/2602.07064v1
• Date: Thu, 05 Feb 2026 14:04:51 GMT
• Title: Exploring Physical Intelligence Emergence via Omni-Modal Architecture and Physical Data Engine
• Abstract summary: We present OmniFysics, a compact omni-modal model that unifies understanding across images, audio, video, and text.<n>To inject explicit physical knowledge, we build a physical data engine with two components.<n>Experiments show competitive performance on standard multimodal benchmarks and improved results on physics-oriented evaluations.
• Score: 50.62040226184694
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physical understanding remains brittle in omni-modal models because key physical attributes are visually ambiguous and sparsely represented in web-scale data. We present OmniFysics, a compact omni-modal model that unifies understanding across images, audio, video, and text, with integrated speech and image generation. To inject explicit physical knowledge, we build a physical data engine with two components. FysicsAny produces physics-grounded instruction--image supervision by mapping salient objects to verified physical attributes through hierarchical retrieval over a curated prototype database, followed by physics-law--constrained verification and caption rewriting. FysicsOmniCap distills web videos via audio--visual consistency filtering to generate high-fidelity video--instruction pairs emphasizing cross-modal physical cues. We train OmniFysics with staged multimodal alignment and instruction tuning, adopt latent-space flow matching for text-to-image generation, and use an intent router to activate generation only when needed. Experiments show competitive performance on standard multimodal benchmarks and improved results on physics-oriented evaluations.

Related papers

• OmniPhys: Knowledge-Graph-Driven Benchmarking and Collective Optimization for Physical Commonsense in Text-to-Image Generation [63.27323042387232]
  We introduce OmniPhys, a rigorous benchmark of 1,551 samples grounded in a Physical Knowledge Graph.<n>By aligning PhET simulations with standard curricula, OmniPhys operationalizes a knowledge-to-scenario pipeline that performs diagnostic stress tests.<n>We propose OmniPrompt, an iterative framework that treats physical feedback as a discrete optimization problem.
  arXiv  Detail & Related papers  (2026-07-28T12:27:25Z)
• VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation [24.373968967561993]
  VIPER is a framework for reference-guided image-to-video generation.<n>It uses a Multimodal Large Language Model to extract reference-derived physical cues.<n>It achieves stronger reference-video physical similarity and higher human preference than representative video generation.
  arXiv  Detail & Related papers  (2026-07-26T05:55:49Z)
• Learning Explicit Physical Parameter Control and Benchmarking for Video Generation [21.331326205371628]
  We present PhyParam, a physics-guided image-to-video diffusion model that conditions on object-level forces, masses, friction, restitution, and scene-level gravity.<n>We also establish PhyParam-Bench, a benchmark for physical-law consistency in image-to-video generation.
  arXiv  Detail & Related papers  (2026-07-21T10:06:55Z)
• PhyCo: Learning Controllable Physical Priors for Generative Motion [55.59209981836171]
  We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
  arXiv  Detail & Related papers  (2026-04-30T17:53:03Z)
• Beyond Semantics: Uncovering the Physics of Fakes via Universal Physical Descriptors for Cross-Modal Synthetic Detection [34.95491424781195]
  We propose a novel feature selection algorithm that identifies five core physical features that exhibit consistent discnative power across all tested datasets.<n>Our method achieves state-of-the-art performance on multiple Genimage benchmarks, with near-perfect accuracy (99.8%) on datasets such as Wukong and SDv1.4.
  arXiv  Detail & Related papers  (2026-04-06T11:50:29Z)
• MMPhysVideo: Scaling Physical Plausibility in Video Generation via Joint Multimodal Modeling [55.72785604682579]
  MMPhysVideo is a framework to scale physical plausibility in video generation in joint multimodal modeling.<n>MMPhysVideo consistently improves physical plausibility and visual quality over advanced models across various benchmarks.
  arXiv  Detail & Related papers  (2026-04-03T07:32:24Z)
• QuPAINT: Physics-Aware Instruction Tuning Approach to Quantum Material Discovery [12.888415301529891]
  Characterizing two-dimensional quantum materials from optical microscopy images is challenging due to subtle layer-dependent contrast, limited labeled data, and significant variation across laboratories and imaging setups.<n>This work presents a new physics-aware multimodal framework that addresses these limitations from both the data and model perspectives.<n>We first present Synthia, a physics-based synthetic data generator that simulates realistic optical responses of quantum material flakes under thin-film interference.<n>We introduce QMat-Instruct, the first large-scale instruction dataset for quantum materials, comprising multimodal, physics-informed question-answer pairs designed to teach Multimodal Large Language Models (ML
  arXiv  Detail & Related papers  (2026-02-19T15:44:41Z)
• PAVAS: Physics-Aware Video-to-Audio Synthesis [58.746986798623084]
  We present Physics-Aware Video-to-Audio Synthesis (PAVAS), a method that incorporates physical reasoning into a latent diffusion-based V2A generation.<n>We show that PAVAS produces physically plausible and perceptually coherent audio, outperforming existing V2A models in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2025-12-09T06:28:50Z)
• VibraVerse: A Large-Scale Geometry-Acoustics Alignment Dataset for Physically-Consistent Multimodal Learning [17.790063818997975]
  VibraVerse is a large-scale dataset that bridges the causal chain from 3D geometry -> physical attributes -> modal parameters -> acoustic signals.<n> CLASP is a contrastive learning framework for cross-modal alignment that preserves the causal correspondence between an object's physical structure and its acoustic response.<n>Built upon VibraVerse, we define a suite of benchmark tasks for geometry-to-sound prediction, sound-guided shape reconstruction, and cross-modal representation learning.
  arXiv  Detail & Related papers  (2025-11-25T15:48:49Z)
• PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection [10.498184571108995]
  We propose PhysCorr, a unified framework for modeling, evaluating, and optimizing physical consistency in video generation.<n>Specifically, we introduce PhysicsRM, the first dual-dimensional reward model that quantifies both intra-object stability and inter-object interactions.<n>Our approach is model-agnostic and scalable, enabling seamless integration into a wide range of video diffusion and transformer-based backbones.
  arXiv  Detail & Related papers  (2025-11-06T02:40:57Z)
• Inferring Dynamic Physical Properties from Video Foundation Models [94.35979242947873]
  We study the task of predicting dynamic physical properties from videos.<n>We consider physical properties that require temporal information to be inferred: elasticity of a bouncing object, viscosity of a flowing liquid, and dynamic friction of an object sliding on a surface.
  arXiv  Detail & Related papers  (2025-10-02T17:59:50Z)
• PhysID: Physics-based Interactive Dynamics from a Single-view Image [1.7214450148288793]
  We present PhysID, that streamlines the creation of physics-based interactive dynamics from a single-view image.<n>We integrate an on-device physics-based engine for physically plausible real-time rendering with user interactions.
  arXiv  Detail & Related papers  (2025-06-21T15:57:58Z)
• PhysGaia: A Physics-Aware Dataset of Multi-Body Interactions for Dynamic Novel View Synthesis [62.283499219361595]
  PhysGaia is a physics-aware dataset specifically designed for Dynamic Novel View Synthesis (DyNVS)<n>Our dataset provides complex dynamic scenarios with rich interactions among multiple objects.<n>PhysGaia will significantly advance research in dynamic view synthesis, physics-based scene understanding, and deep learning models integrated with physical simulation.
  arXiv  Detail & Related papers  (2025-06-03T12:19:18Z)
• Think Before You Diffuse: Infusing Physical Rules into Video Diffusion [55.046699347579455]
  The complexity of real-world motions, interactions, and dynamics introduce great difficulties when learning physics from data.<n>We propose DiffPhy, a generic framework that enables physically-correct and photo-realistic video generation by fine-tuning a pre-trained video diffusion model.
  arXiv  Detail & Related papers  (2025-05-27T18:26:43Z)
• ContPhy: Continuum Physical Concept Learning and Reasoning from Videos [86.63174804149216]
  ContPhy is a novel benchmark for assessing machine physical commonsense. We evaluated a range of AI models and found that they still struggle to achieve satisfactory performance on ContPhy. We also introduce an oracle model (ContPRO) that marries the particle-based physical dynamic models with the recent large language models.
  arXiv  Detail & Related papers  (2024-02-09T01:09:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.