VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System

• URL: http://arxiv.org/abs/2607.27380v1
• Date: Wed, 29 Jul 2026 18:38:23 GMT
• Title: VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System
• Abstract summary: Text-video models struggle to generate physically consistent dynamics the temporal evolution of a scene must be inferred implicitly from a compressed text prompt.<n>Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-cutable or temporally sparse.<n>We introduce VideoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought.
• Score: 77.32035315337522
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.

Related papers

• Generation Is Compression: Zero-Shot Video Coding via Stochastic Rectified Flow [1.8898767128464697]
  emphGenerative Video Codebook Codec (GVCC) is a zero-shot framework that turns a pretrained video generative model into the itself.<n>Building on this unified backbone, we instantiate three conditioning strategies -- emphImage-to-Video (I2V), emphText-to-Video (T2V) and emphFirst-Last-Frame-to-Video (FLF2V)
  arXiv  Detail & Related papers  (2026-03-27T16:33:20Z)
• Plenoptic Video Generation [80.3116444692858]
  We introduce PlenopticDreamer, a framework that synchronizes generative hallucinations to maintain synchronization-temporal memory.<n>The core idea is to train a multi-in-out video-conditioned model in an autoregressive manner.<n>Our training incorporates context-scaling to improve convergence, self-conditioning to hallucinations caused by error accumulation, and a long-video conditioning mechanism to support extended video generation.
  arXiv  Detail & Related papers  (2026-01-08T18:58:32Z)
• Mirage: One-Step Video Diffusion for Photorealistic and Coherent Asset Editing in Driving Scenes [11.72942978943414]
  Vision-centric autonomous driving systems rely on diverse and scalable training data to achieve robust performance.<n>Video object editing offers a promising path for data augmentation.<n>Existing methods often struggle to maintain both high visual fidelity and temporal coherence.<n>Mirage builds upon a text-to-video diffusion prior to ensure temporal consistency across frames.
  arXiv  Detail & Related papers  (2025-12-30T13:40:23Z)
• CoAgent: Collaborative Planning and Consistency Agent for Coherent Video Generation [9.91271343855315]
  CoAgent is a framework for coherent video generation that formulates the process as a plan-synthesize-verify pipeline.<n>A Storyboard Planner decomposes the input into structured shot-level plans with explicit entities, spatial relations, and temporal cues.<n>A Global Context Manager maintains entity-level memory to preserve appearance and identity consistency across shots.<n>A pacing-aware editor refines temporal rhythm and transitions to match the desired narrative flow.
  arXiv  Detail & Related papers  (2025-12-27T09:38:34Z)
• Factorized Video Generation: Decoupling Scene Construction and Temporal Synthesis in Text-to-Video Diffusion Models [76.7535001311919]
  State-of-the-art Text-to-Video (T2V) diffusion models can generate visually impressive results, yet they often fail to compose complex scenes or follow logical temporal instructions.<n>We introduce Factorized Video Generation (FVG), a pipeline that decouples these tasks by decomposing the Text-to-Video generation into three specialized stages.<n>Our approach sets a new state-of-the-art on the T2V CompBench benchmark and significantly improves all tested models on VBench2.
  arXiv  Detail & Related papers  (2025-12-18T10:10:45Z)
• TemCoCo: Temporally Consistent Multi-modal Video Fusion with Visual-Semantic Collaboration [36.255570023185506]
  Existing multi-modal fusion methods apply static frame-based image fusion techniques directly to video fusion tasks.<n>We propose the first video fusion framework that explicitly incorporates temporal modeling with visual-semantic collaboration.
  arXiv  Detail & Related papers  (2025-08-25T09:12:55Z)
• LoViC: Efficient Long Video Generation with Context Compression [68.22069741704158]
  We introduce LoViC, a DiT-based framework trained on million-scale open-domain videos.<n>At the core of our approach is FlexFormer, an expressive autoencoder that jointly compresses video and text into unified latent representations.
  arXiv  Detail & Related papers  (2025-07-17T09:46:43Z)
• VideoComposer: Compositional Video Synthesis with Motion Controllability [52.4714732331632]
  VideoComposer allows users to flexibly compose a video with textual conditions, spatial conditions, and more importantly temporal conditions. We introduce the motion vector from compressed videos as an explicit control signal to provide guidance regarding temporal dynamics. In addition, we develop a Spatio-Temporal Condition encoder (STC-encoder) that serves as a unified interface to effectively incorporate the spatial and temporal relations of sequential inputs.
  arXiv  Detail & Related papers  (2023-06-03T06:29:02Z)
• Towards Smooth Video Composition [59.134911550142455]
  Video generation requires consistent and persistent frames with dynamic content over time. This work investigates modeling the temporal relations for composing video with arbitrary length, from a few frames to even infinite, using generative adversarial networks (GANs) We show that the alias-free operation for single image generation, together with adequately pre-learned knowledge, brings a smooth frame transition without compromising the per-frame quality.
  arXiv  Detail & Related papers  (2022-12-14T18:54:13Z)
• Autoencoding Video Latents for Adversarial Video Generation [0.0]
  AVLAE is a two stream latent autoencoder where the video distribution is learned by adversarial training. We demonstrate that our approach learns to disentangle motion and appearance codes even without the explicit structural composition in the generator.
  arXiv  Detail & Related papers  (2022-01-18T11:42:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.