PAI-Actor: Cinematic Multi-Character Replacement in Dynamic Scenes

• URL: http://arxiv.org/abs/2609.05918v1
• Date: Sat, 05 Sep 2026 06:27:27 GMT
• Title: PAI-Actor: Cinematic Multi-Character Replacement in Dynamic Scenes
• Abstract summary: We present PAI-Actor, a cinematic multi-character animation framework for character replacement in dynamic movie scenes.<n>Our goal is to replace and animate multiple characters within real video clips while preserving the original scene dynamics, camera motion, and background content.<n>Experiments show that PAI-Actor enables high-fidelity multi-character animation with strong scene consistency, cinematic visual quality, and efficient long-form generation.
• Score: 51.8637101384136
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present PAI-Actor, a cinematic multi-character animation framework for character replacement in dynamic movie scenes. Unlike conventional animation systems that mainly drive a single static image or a single subject, our goal is to replace and animate multiple characters within real video clips while preserving the original scene dynamics, camera motion, and background content. This setting is particularly challenging because the generated characters must remain consistent with the source performance in motion and interaction, while also matching the surrounding background in lighting, shadow, composition, and overall cinematic appearance. To address this, we formulate multi-character animation as a structure-guided human recovery problem and build a movie-driven training pipeline from high-quality film data. Furthermore, to support practical cinematic production, we introduce a bidirectional-to-autoregressive distillation framework: we first train a bidirectional diffusion transformer for high-quality short-clip generation at 1080P resolution, and then distill it into an autoregressive video-to-video model for efficient inference and longer video generation. Experiments show that PAI-Actor enables high-fidelity multi-character animation with strong scene consistency, cinematic visual quality, and efficient long-form generation.

Related papers

• Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaboration [61.98029663481308]
  Soap2Soap is a multi-agent framework that enforces long-term visual consistency through a DualBridge mechanism.<n>A closed-loop verification agent audits identity, stability, and alignment to trigger selective regeneration.
  arXiv  Detail & Related papers  (2026-05-17T12:38:21Z)
• MVAnimate: Enhancing Character Animation with Multi-View Optimization [55.4217617472079]
  We introduce MVAnimate, a novel framework that synthesizes both 2D and 3D information of dynamic figures based on multi-view prior information.<n>Our approach leverages multi-view prior information to produce temporally consistent and spatially coherent animation outputs.
  arXiv  Detail & Related papers  (2026-02-09T14:55:21Z)
• CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generation [65.03946626081036]
  We present CineScene, a framework that leverages implicit 3D-aware scene representation for cinematic video generation.<n>CineScene achieves state-of-the-art performance in scene-consistent cinematic video generation.
  arXiv  Detail & Related papers  (2026-02-06T18:59:24Z)
• Wan-S2V: Audio-Driven Cinematic Video Generation [53.59046547320306]
  Current state-of-the-art (SOTA) methods for audio-driven character animation demonstrate promising performance for scenarios involving speech and singing.<n>We propose an audio-driven model, which we refere to as Wan-S2V, built upon Wan.
  arXiv  Detail & Related papers  (2025-08-26T02:51:31Z)
• Animate-X++: Universal Character Image Animation with Dynamic Backgrounds [32.04255747303296]
  Animate-X++ is a universal animation framework based on DiT for various character types, including anthropomorphic characters.<n>To enhance motion representation, we introduce the Pose Indicator, which captures comprehensive motion pattern from the driving video through both implicit and explicit manner.<n>For the second challenge, we introduce a multi-task training strategy that jointly trains the animation and TI2V tasks.
  arXiv  Detail & Related papers  (2025-08-13T03:11:28Z)
• FairyGen: Storied Cartoon Video from a Single Child-Drawn Character [15.701180508477679]
  We propose FairyGen, an automatic system for generating story-driven cartoon videos from a single child's drawing.<n>Unlike previous storytelling methods, FairyGen explicitly disentangles character modeling from stylized background generation.<n>Our system produces animations that are stylistically faithful, narratively structured natural motion.
  arXiv  Detail & Related papers  (2025-06-26T13:58:16Z)
• DreamDance: Animating Character Art via Inpainting Stable Gaussian Worlds [64.53681498600065]
  DreamDance is an animation framework capable of producing stable, consistent character and scene motion conditioned on precise camera trajectories.<n>We train a pose-aware video inpainting model that injects the dynamic character into the scene video while enhancing background quality.
  arXiv  Detail & Related papers  (2025-05-30T15:54:34Z)
• SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers [30.06494915665044]
  We present SkyReels-A1, a framework built upon video diffusion Transformer to facilitate portrait image animation.<n>SkyReels-A1 capitalizes on the strong generative capabilities of video DiT, enhancing facial motion transfer precision, identity retention, and temporal coherence.<n>It is highly applicable to domains such as virtual avatars, remote communication, and digital media generation.
  arXiv  Detail & Related papers  (2025-02-15T16:08:40Z)
• One-Shot Learning Meets Depth Diffusion in Multi-Object Videos [0.0]
  This paper introduces a novel depth-conditioning approach that enables the generation of coherent and diverse videos from just a single text-video pair. Our method fine-tunes the pre-trained model to capture continuous motion by employing custom-designed spatial and temporal attention mechanisms. During inference, we use the DDIM inversion to provide structural guidance for video generation.
  arXiv  Detail & Related papers  (2024-08-29T16:58:10Z)
• Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling [77.08568533331206]
  We propose a novel multi-condition guided framework for character image animation.<n>We employ several well-designed input modules to enhance the implicit decoupling capability of the model.<n>Our method excels in generating high-quality character animations, especially in scenarios of complex backgrounds and multiple characters.
  arXiv  Detail & Related papers  (2024-06-05T08:03:18Z)
• AniClipart: Clipart Animation with Text-to-Video Priors [28.76809141136148]
  We introduce AniClipart, a computational system that converts static animations into high-quality animations guided by text-to-video prior.<n>By differentiable As-Rigid-As-Possible (ARAP) shape deformation and differentiable rendering, AniClipart can be end-to-end optimized while maintaining rigidity.
  arXiv  Detail & Related papers  (2024-04-18T17:24:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.