MorphoSim: An Interactive, Controllable, and Editable Language-guided 4D World Simulator

• URL: http://arxiv.org/abs/2510.04390v1
• Date: Sun, 05 Oct 2025 22:55:17 GMT
• Title: MorphoSim: An Interactive, Controllable, and Editable Language-guided 4D World Simulator
• Abstract summary: We introduce MorphoSim, a language guided framework that generates 4D scenes with multi-view consistency and object-level controls.<n>The framework integrates trajectory-guided generation with feature field dis tillation, allowing edits to be applied interactively without full re-generation.
• Score: 42.06769509897601
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models that support controllable and editable spatiotemporal environments are valuable for robotics, enabling scalable training data, repro ducible evaluation, and flexible task design. While recent text-to-video models generate realistic dynam ics, they are constrained to 2D views and offer limited interaction. We introduce MorphoSim, a language guided framework that generates 4D scenes with multi-view consistency and object-level controls. From natural language instructions, MorphoSim produces dynamic environments where objects can be directed, recolored, or removed, and scenes can be observed from arbitrary viewpoints. The framework integrates trajectory-guided generation with feature field dis tillation, allowing edits to be applied interactively without full re-generation. Experiments show that Mor phoSim maintains high scene fidelity while enabling controllability and editability. The code is available at https://github.com/eric-ai-lab/Morph4D.

Related papers

• SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators [61.95179776027961]
  SyncWorld is an action-conditioned world model that serves as a zero-shot simulator across unseen environments without any additional training.<n>Training with visual calibration contexts teaches the model to interpret actions through visual evidence.<n>Experiments show that SyncWorld can accurately simulate action outcomes in previously unseen settings.
  arXiv  Detail & Related papers  (2026-09-08T17:59:47Z)
• Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models [57.59888019135289]
  Hash-Atlas network reformulates 3D scene editing as operations on 2D atlas images.<n>We introduce a dialogue-based 3D scene editing approach, termed CE3D++.<n>Results demonstrate that CE3D++ effectively integrates multiple visual models to achieve diverse visual editing effects.
  arXiv  Detail & Related papers  (2026-08-29T08:25:58Z)
• 4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation [16.13796756164597]
  Embodied agents need environments that are visually diverse, physically interactive, and changing over time.<n>We present 4D Synth, a controllable procedural system that turns a natural-language description, a blueprint mask, or a single photograph into an editable 4D environment.<n>Multiple scene routes share one geometry-grounded representation, so the same pipeline handles animation, camera planning, rendering, and task generation.
  arXiv  Detail & Related papers  (2026-08-27T10:49:20Z)
• Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents [42.18672329354181]
  We introduce textitAgentic Real2Sim, a framework for generalized physical world modeling with vision-language agents.<n>We evaluate Agentic Real2Sim on rigid-object manipulation, deformable-object interaction, and humanoid motion scenes.<n>The framework's agentic decisions can be driven by an open-weight VLM backend at a small fraction of the cost of frontier models.
  arXiv  Detail & Related papers  (2026-07-21T15:23:38Z)
• SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation [50.175987640001644]
  We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text.<n>We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes.
  arXiv  Detail & Related papers  (2026-07-02T06:29:18Z)
• WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes [15.223046632151364]
  We present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes.<n>WorldAct uses a multimodal agent to guide scene decomposition, identify actionable objects, reconstruct geometrically aligned object-level meshes for interaction, and restore the residual background via 3D inpainting.<n> Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.
  arXiv  Detail & Related papers  (2026-05-15T10:56:23Z)
• SceneFoundry: Generating Interactive Infinite 3D Worlds [22.60801815197924]
  SceneFoundry is a language-guided diffusion framework that generates apartment-scale 3D worlds with functionally articulated furniture.<n>Our framework generates structurally valid, semantically coherent, and functionally interactive environments across diverse scene types and conditions.
  arXiv  Detail & Related papers  (2026-01-09T14:33:10Z)
• Animate Any Character in Any World [61.112404900403284]
  We introduce AniX, leveraging the realism and structural grounding of static world generation.<n>Users can provide a 3DGS scene and a character, then direct the character through natural language to perform diverse behaviors.<n>AiX synthesizes temporally coherent video clips that preserve visual fidelity with the provided scene and character.
  arXiv  Detail & Related papers  (2025-12-18T18:59:18Z)
• VHOI: Controllable Video Generation of Human-Object Interactions from Sparse Trajectories via Motion Densification [65.15340059997273]
  VHOI is a framework for creating realistic human-object interactions in video.<n>We introduce a novel HOI-aware motion representation that uses color encodings to distinguish not only human and object motion, but also body-part-specific dynamics.<n> Experiments demonstrate state-of-the-art results in controllable HOI video generation.
  arXiv  Detail & Related papers  (2025-12-10T13:40:24Z)
• Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model [19.937724706042804]
  Hunyuan-GameCraft-2 is a new paradigm of instruction-driven interaction for generative game world modeling.<n>Our model allows users to control game video contents through natural language prompts, keyboard, or mouse signals.<n>Our model generates temporally coherent and causally grounded interactive game videos.
  arXiv  Detail & Related papers  (2025-11-28T18:26:39Z)
• Human-Object Interaction via Automatically Designed VLM-Guided Motion Policy [30.43930233035367]
  We introduce the first unified physics-based HO framework that leverages Vision-Language Models (VLMs)<n>We introduce VLM-Guided Relative Movement Dynamics (RMD), a fine-temporal bipartite motion representation that automatically constructs goal states and reward functions for reinforcement learning.<n>To support our methodology, we present Interplay, a novel dataset with thousands of long-horizon static and dynamic interaction plans.
  arXiv  Detail & Related papers  (2025-03-24T05:18:04Z)
• SCENIC: Scene-aware Semantic Navigation with Instruction-guided Control [36.22743674288336]
  SCENIC is a diffusion model designed to generate human motion that adapts to dynamic terrains within virtual scenes.<n>Our system achieves seamless transitions between different motion styles while maintaining scene constraints.<n>Our code, dataset, and models will be released at urlhttps://virtualhumans.mpi-inf.mpg.de/scenic/.
  arXiv  Detail & Related papers  (2024-12-20T08:25:15Z)
• LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control [43.25983939699873]
  This paper scales object-level reconstruction to complex scenes, advancing interactive scene reconstruction.<n>We propose LiveScene, a scene-level language-embedded interactive radiance field that efficiently reconstructs and controls multiple objects.
  arXiv  Detail & Related papers  (2024-06-23T07:26:13Z)
• Generating Human Interaction Motions in Scenes with Text Control [66.74298145999909]
  We present TeSMo, a method for text-controlled scene-aware motion generation based on denoising diffusion models. Our approach begins with pre-training a scene-agnostic text-to-motion diffusion model. To facilitate training, we embed annotated navigation and interaction motions within scenes.
  arXiv  Detail & Related papers  (2024-04-16T16:04:38Z)
• TC4D: Trajectory-Conditioned Text-to-4D Generation [94.90700997568158]
  We propose TC4D: trajectory-conditioned text-to-4D generation, which factors motion into global and local components. We learn local deformations that conform to the global trajectory using supervision from a text-to-video model. Our approach enables the synthesis of scenes animated along arbitrary trajectories, compositional scene generation, and significant improvements to the realism and amount of generated motion.
  arXiv  Detail & Related papers  (2024-03-26T17:55:11Z)
• Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents [49.11254369459406]
  ChatSim is the first system that enables editable photo-realistic 3D driving scene simulations via natural language commands with external digital assets. ChatSim employs a novel multi-camera neural field method to generate photo-realistic outcomes.
  arXiv  Detail & Related papers  (2024-02-08T15:26:28Z)
• VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models [38.503337052122234]
  Large language models (LLMs) are shown to possess a wealth of actionable knowledge that can be extracted for robot manipulation. We aim to synthesize robot trajectories for a variety of manipulation tasks given an open-set of instructions and an open-set of objects. We demonstrate how the proposed framework can benefit from online experiences by efficiently learning a dynamics model for scenes that involve contact-rich interactions.
  arXiv  Detail & Related papers  (2023-07-12T07:40:48Z)
• iGibson, a Simulation Environment for Interactive Tasks in Large Realistic Scenes [54.04456391489063]
  iGibson is a novel simulation environment to develop robotic solutions for interactive tasks in large-scale realistic scenes. Our environment contains fifteen fully interactive home-sized scenes populated with rigid and articulated objects. iGibson features enable the generalization of navigation agents, and that the human-iGibson interface and integrated motion planners facilitate efficient imitation learning of simple human demonstrated behaviors.
  arXiv  Detail & Related papers  (2020-12-05T02:14:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.