PhysAgent: Automating Physics-Based 4D Synthesis via Trajectory-Grounded Multi-Agent Feedback

• URL: http://arxiv.org/abs/2606.08688v1
• Date: Sun, 07 Jun 2026 15:42:56 GMT
• Title: PhysAgent: Automating Physics-Based 4D Synthesis via Trajectory-Grounded Multi-Agent Feedback
• Abstract summary: PhysAgent is a simulator-in-the-loop multi-agent framework for automated, physically grounded 4D synthesis.<n>It generates stable, diverse physical scenes from arbitrary multimodal prompts, significantly outperforming existing baselines in both generation diversity and physical accuracy.
• Score: 20.18865007569388
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Achieving fully automated, physically plausible 3D motion synthesis is a core objective in graphics and generative AI. However, configuring complex environmental force fields still relies entirely on manual expert intervention, creating a severe bottleneck for large-scale simulation data generation. Existing automated methods primarily focus on material optimization and exhibit severe modality gaps and technical flaws when applied to the vastly more complex force field optimization space: naive Large Language Models (LLMs) lack underlying simulation feedback, causing severe physical inaccuracies, while traditional Score Distillation Sampling (SDS) suffers from sluggish gradients, local optima entrapment, and a mathematical inability to dynamically switch discrete force fields. To address this, we propose PhysAgent, the first simulator-in-the-loop multi-agent framework that leverages multimodal inputs for automated, physically grounded 4D synthesis. By decoupling intrinsic materials from extrinsic dynamics, PhysAgent utilizes a Semantic Agent equipped with an externalized Force Field Skill module to master simulation rules and generate valid initializations. Subsequently, the Refine Agents, driven by Trajectory-Grounded Multi-Agent Feedback, leverage vision foundation models to extract dense point trajectories from rendered frames. By converting these explicit motion trajectories into structured textual descriptors, the agent harnesses LLM commonsense reasoning to execute zero-shot macroscopic leaps, effectively escaping local optima and dynamically switching discrete force fields. Extensive experiments demonstrate that PhysAgent rapidly generates stable, diverse physical scenes from arbitrary multimodal prompts, significantly outperforming existing baselines in both generation diversity and physical accuracy.

Related papers

• PhysMAS: Physics-Grounded Multi-Agent Synthesis of Compositional 4D Gaussians [34.993124532814186]
  PhysMAS is a physics-grounded multi-agent framework for dynamic scene generation.<n>It invokes solver-aware skills to bind identities and profiles to per-particle MPM fields and execute all objects in a shared domain.<n>It achieves better semantic alignment and perceived physical plausibility while requiring less runtime.
  arXiv  Detail & Related papers  (2026-09-07T08:07:05Z)
• PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation [68.13035350819901]
  PhysAgent is a framework that closes the loop among physical program generation, physics simulation, stage-specific verification, and targeted program repair.<n>Our framework produces more physically plausible videos, achieves better prompt alignment, and generalizes more effectively across diverse physical scenarios.
  arXiv  Detail & Related papers  (2026-07-17T08:55:09Z)
• PhyCo: Learning Controllable Physical Priors for Generative Motion [55.59209981836171]
  We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
  arXiv  Detail & Related papers  (2026-04-30T17:53:03Z)
• Resonance4D: Frequency-Domain Motion Supervision for Preset-Free Physical Parameter Learning in 4D Dynamic Physical Scene Simulation [53.141181758015186]
  We present Resonance4D, a physics-driven 4D dynamic simulation framework that couples 3D Gaussian Splatting with the Material Point Method.<n>Our key insight is that dynamic consistency can be enforced without dense temporal generation by jointly constraining motion in complementary domains.<n>We show that Resonance4D achieves strong physical fidelity and motion consistency while reducing peak GPU memory from over 35,GB to around 20,GB.
  arXiv  Detail & Related papers  (2026-04-02T13:00:22Z)
• RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids [20.796118584632904]
  We present a unified latent-driven framework that bridges natural language and whole-body locomotion.<n>Our framework provides a practical path toward deployable text-guided humanoid intelligence.
  arXiv  Detail & Related papers  (2026-03-18T17:02:56Z)
• Physically Accurate Rigid-Body Dynamics in Particle-Based Simulation [38.44669833143752]
  Particle-based simulators offer a compelling alternative to current simulators.<n>Position-based dynamics (PBD) is a popular solver known for its computational efficiency and visual plausibility.<n>We introduce PBD-R, a revised PBD formulation that enforces physically accurate rigid-body dynamics.
  arXiv  Detail & Related papers  (2026-03-15T22:09:42Z)
• MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twins [25.25391483526521]
  MotionAnymesh is a framework that seamlessly transforms unstructured static meshes into simulation-ready digital twins.<n>Our method features a kinematic-aware part segmentation module that grounds VLM reasoning with explicit SP4D physical priors.
  arXiv  Detail & Related papers  (2026-03-13T12:30:42Z)
• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimization [56.17833729527066]
  We propose FastPhysGS, a framework for physics-based dynamic 3DGS simulation.<n>FastPhysGS achieves high-fidelity physical simulation in 1 minute using only 7 GB runtime memory.
  arXiv  Detail & Related papers  (2026-02-02T07:00:42Z)
• URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Model [76.08429266631823]
  We propose an end-to-end automatic reconstruction framework based on a 3D multimodal large language model (MLLM)<n>URDF-Anything utilizes an autoregressive prediction framework based on point-cloud and text multimodal input to jointly optimize geometric segmentation and kinematic parameter prediction.<n> Experiments on both simulated and real-world datasets demonstrate that our method significantly outperforms existing approaches.
  arXiv  Detail & Related papers  (2025-11-02T13:45:51Z)
• PhyMAGIC: Physical Motion-Aware Generative Inference with Confidence-guided LLM [17.554471769834453]
  We present PhyMAGIC, a training-free framework that generates physically consistent motion from a single image.<n>PhyMAGIC integrates a pre-trained image-to-video diffusion model, confidence-guided reasoning via LLMs, and a differentiable physics simulator.<n> Comprehensive experiments demonstrate that PhyMAGIC outperforms state-of-the-art video generators and physics-aware baselines.
  arXiv  Detail & Related papers  (2025-05-22T09:40:34Z)
• PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics [89.81550748680245]
  We introduce a new differentiable physics benchmark called PasticineLab. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark.
  arXiv  Detail & Related papers  (2021-04-07T17:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.