DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos

• URL: http://arxiv.org/abs/2605.09586v1
• Date: Sun, 10 May 2026 14:55:54 GMT
• Title: DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos
• Abstract summary: We present DeformMaster, a video-derived interactive physics--neural world model that turns real interaction videos into an online model of deformable objects.<n> Experiments on real-world deformable-object sequences demonstrate DeformMaster's ability to roll out future dynamics and render dynamic appearance.
• Score: 13.695348404320475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models for deformable objects should recover not only geometry and appearance, but also underlying physical dynamics, interaction grounding, and material behavior. Learning such a model from real videos is challenging because deformable linear, planar, and volumetric objects evolve under high-dimensional deformation, noisy interactions, and complex material response. The model must therefore infer a physical state from visual observations, roll it forward under new interactions, and render the resulting dynamics with high visual fidelity. We present DeformMaster, a video-derived interactive physics--neural world model that turns real interaction videos into an online interactive model of deformable objects within a unified dynamics-and-appearance framework. DeformMaster preserves structured physical rollout while using a neural residual to compensate for unmodeled effects, grounds sparse hand motion as distributed compliant actuator for hand--continuum interaction, represents material response with spatially varying constitutive experts, and drives high-fidelity 4D appearance from the predicted physical evolution. Experiments on real-world deformable-object sequences demonstrate DeformMaster's ability to roll out future dynamics and render dynamic appearance, outperforming state-of-the-art baselines while supporting novel action rollout, material-parameter variation, and dynamic novel-view synthesis.

Related papers

• PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling [22.56145247266043]
  We propose a video-driven framework for learning and simulating the underlying physics of real deformable objects.<n> PhysReal integrates a spatially varying hybrid expert-neural model with a differentiable MPM simulator and 3DGS.<n>Experiments on diverse deformable-object interactions demonstrate that PhysReal achieves superior performance in dynamic reconstruction and future-state prediction.
  arXiv  Detail & Related papers  (2026-09-07T14:11:57Z)
• Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videos [2.5608298830470013]
  Inferring object dynamics from visual observations is essential for intelligent agents to reason about and interact with the physical world.<n>We propose Analytic Dynamics, a feed-forward dynamics inference framework that introduces an intermediate physics-grounded dynamics representation.
  arXiv  Detail & Related papers  (2026-08-31T16:08:04Z)
• Self-Supervised Learning of Structured Dynamics from Videos [76.23834498074805]
  We study whether structured motion representations can be recovered from frozen features of a pretrained image vision transformer.<n>We propose the Structured Dynamics Model (SDM), which explicitly separates the dominant source of temporal change from residual dynamics.<n>We evaluate SDM on ProbeMotion, a new evaluation suite spanning synthetic and real videos with camera motion, object motion, and combined dynamics.
  arXiv  Detail & Related papers  (2026-07-23T17:55:07Z)
• Learning Explicit Physical Parameter Control and Benchmarking for Video Generation [21.331326205371628]
  We present PhyParam, a physics-guided image-to-video diffusion model that conditions on object-level forces, masses, friction, restitution, and scene-level gravity.<n>We also establish PhyParam-Bench, a benchmark for physical-law consistency in image-to-video generation.
  arXiv  Detail & Related papers  (2026-07-21T10:06:55Z)
• PhyCo: Learning Controllable Physical Priors for Generative Motion [55.59209981836171]
  We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
  arXiv  Detail & Related papers  (2026-04-30T17:53:03Z)
• Phantom: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamics [12.143531149918674]
  We propose a Physics-Infused Video Generation model that jointly models the visual content and latent physical dynamics.<n>Conditioned on observed video frames and inferred physical states, Phantom jointly predicts latent physical dynamics and generates future video frames.<n>By integrating the inference of physical-aware video representation directly into the video generation process, Phantom produces video sequences that are both visually realistic and physically consistent.
  arXiv  Detail & Related papers  (2026-04-09T17:48:46Z)
• PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis [52.905353023326306]
  We propose PhysWorld, a framework that synthesizes physically plausible and diverse demonstrations to learn efficient world models.<n>Experiments show that PhysWorld has competitive performance while enabling inference speeds 47 times faster than the recent state-of-the-art method, i.e., PhysTwin.
  arXiv  Detail & Related papers  (2025-10-24T13:25:39Z)
• Half-Physics: Enabling Kinematic 3D Human Model with Physical Interactions [89.88331682333198]
  We introduce a novel approach that embeds SMPL-X into a tangible entity capable of dynamic physical interactions with its surroundings.<n>Our approach maintains kinematic control over inherent SMPL-X poses while ensuring physically plausible interactions with scenes and objects.<n>Unlike reinforcement learning-based methods, which demand extensive and complex training, our half-physics method is learning-free and generalizes to any body shape and motion.
  arXiv  Detail & Related papers  (2025-07-31T17:58:33Z)
• PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos [21.441062722848265]
  PhysTwin is a novel framework that uses sparse videos of dynamic objects under interaction to produce a photo- and physically realistic, real-time interactive replica.<n>Our approach centers on two key components: (1) a physics-informed representation that combines spring-mass models for realistic physical simulation, and generative shape models for geometry, and Gaussian splats for rendering.<n>Our method integrates an inverse physics framework with visual perception cues, enabling high-fidelity reconstruction even from partial, occluded, and limited viewpoints.
  arXiv  Detail & Related papers  (2025-03-23T07:49:19Z)
• InterDyn: Controllable Interactive Dynamics with Video Diffusion Models [50.38647583839384]
  We propose InterDyn, a framework that generates videos of interactive dynamics given an initial frame and a control signal encoding the motion of a driving object or actor.<n>Our key insight is that large video generation models can act as both neurals and implicit physics simulators'', having learned interactive dynamics from large-scale video data.
  arXiv  Detail & Related papers  (2024-12-16T13:57:02Z)
• OCK: Unsupervised Dynamic Video Prediction with Object-Centric Kinematics [22.119612406160073]
  We present OCK, a dynamic video prediction model leveraging object-centric kinematics and object slots.<n>We introduce a novel component named Object Kinematics that comprises explicit object motions.<n>Our model demonstrates superior performance in complex scenes with intricate object attributes and motions.
  arXiv  Detail & Related papers  (2024-04-29T04:47:23Z)
• PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation [62.53760963292465]
  PhysDreamer is a physics-based approach that endows static 3D objects with interactive dynamics. We present our approach on diverse examples of elastic objects and evaluate the realism of the synthesized interactions through a user study.
  arXiv  Detail & Related papers  (2024-04-19T17:41:05Z)
• Real-time Deep Dynamic Characters [95.5592405831368]
  We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing. We show that our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches.
  arXiv  Detail & Related papers  (2021-05-04T23:28:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.