InstanceAnimator: Multi-Instance Sketch Video Colorization

• URL: http://arxiv.org/abs/2603.25357v1
• Date: Thu, 26 Mar 2026 12:01:46 GMT
• Title: InstanceAnimator: Multi-Instance Sketch Video Colorization
• Abstract summary: We propose InstanceAnimator, a novel Diffusion Transformer framework for multi-instance sketch video colorization.<n>Existing methods suffer from three core limitations: inflexible user control due to heavy reliance on single reference frames, poor instance controllability, and degraded detail fidelity in fine-grained regions.<n>We show that InstanceAnimator achieves superior multi-instance colorization with enhanced user control, high visual quality, and strong instance consistency.
• Score: 54.60341071606437
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose InstanceAnimator, a novel Diffusion Transformer framework for multi-instance sketch video colorization. Existing methods suffer from three core limitations: inflexible user control due to heavy reliance on single reference frames, poor instance controllability leading to misalignment in multi-character scenarios, and degraded detail fidelity in fine-grained regions. To address these challenges, we introduce three corresponding innovations. First, a Canvas Guidance Condition eliminates workflow fragmentation by allowing free placement of reference elements and background, enabling unprecedented user flexibility. Second, an Instance Matching Mechanism resolves misalignment by integrating instance features with the sketches, ensuring precise control over multiple characters. Third, an Adaptive Decoupled Control Module enhances detail fidelity by injecting semantic features from characters, backgrounds, and text conditions into the diffusion process. Extensive experiments demonstrate that InstanceAnimator achieves superior multi-instance colorization with enhanced user control, high visual quality, and strong instance consistency.

Related papers

• MultiAnimate: A Unified Framework for Controllable Multi-Character Animation [76.56813950148235]
  MultiAnimate is a framework that enables concurrent animation of multiple characters within a shared environment.<n>We introduce an identity-specific reference net that enables appearance extraction from multiple reference images.<n>Second, we implement an identity-aware pose encoder to address the character-pose binding challenge.<n>Third, we introduce an interaction guider module that enhances the framework's capability to handle complex inter-character interactions.
  arXiv  Detail & Related papers  (2026-07-15T03:33:52Z)
• Tri-Prompting: Video Diffusion with Unified Control over Scene, Subject, and Motion [69.64308172688368]
  Tri-Prompting is a unified framework that integrates scene composition, multi-view subject consistency, and motion control.<n>Results demonstrate that Tri-Prompting significantly outperforms specialized baselines such as Phantom and DaS in multi-view subject identity, 3D consistency, and motion accuracy.
  arXiv  Detail & Related papers  (2026-03-16T17:59:05Z)
• Canvas-to-Image: Compositional Image Generation with Multimodal Controls [51.44122945214702]
  We introduce Canvas-to-Image, a unified framework that consolidates heterogeneous controls into a single canvas interface.<n>Our key idea is to encode diverse control signals into a single composite canvas image that the model can interpret for integrated visual-spatial reasoning.
  arXiv  Detail & Related papers  (2025-11-26T18:59:56Z)
• Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizer [58.94607850223466]
  We present Color3D, a highly adaptable framework for colorizing both static and dynamic 3D scenes from monochromatic inputs.<n>Our approach is able to preserve color diversity and steerability while ensuring cross-view and cross-time consistency.
  arXiv  Detail & Related papers  (2025-10-11T10:21:19Z)
• MultiCOIN: Multi-Modal COntrollable Video INbetweening [46.37499813275259]
  We introduce MultiCOIN, a video inbetweening framework that allows multi-modal controls.<n>To ensure compatibility between DiT and our multi-modal controls, we map all motion controls into a common sparse representation.<n>We also propose a stage-wise training strategy to ensure that our model learns the multi-modal controls smoothly.
  arXiv  Detail & Related papers  (2025-10-09T17:59:27Z)
• Ctrl-VI: Controllable Video Synthesis via Variational Inference [62.79016502243712]
  Ctrl-VI is a video synthesis method that generates samples with high controllability for specified elements.<n>We show that our method produces samples with improved controllability, diversity, and 3D consistency compared to prior works.
  arXiv  Detail & Related papers  (2025-10-09T01:48:16Z)
• Generating Compositional Scenes via Text-to-image RGBA Instance Generation [82.63805151691024]
  Text-to-image diffusion generative models can generate high quality images at the cost of tedious prompt engineering. We propose a novel multi-stage generation paradigm that is designed for fine-grained control, flexibility and interactivity. Our experiments show that our RGBA diffusion model is capable of generating diverse and high quality instances with precise control over object attributes.
  arXiv  Detail & Related papers  (2024-11-16T23:44:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.