SketchColour: Channel Concat Guided DiT-based Sketch-to-Colour Pipeline for 2D Animation

• URL: http://arxiv.org/abs/2507.01586v1
• Date: Wed, 02 Jul 2025 10:57:16 GMT
• Title: SketchColour: Channel Concat Guided DiT-based Sketch-to-Colour Pipeline for 2D Animation
• Authors: Bryan Constantine Sadihin, Michael Hua Wang, Shei Pern Chua, Hang Su,
• Abstract summary: We present SketchColour, the first sketch-to-colour pipeline for 2D animation built on a diffusion transformer (DiT) backbone.<n>We replace the conventional U-Net denoiser with a DiT-style architecture and injecting sketch information via lightweight channel-concatenation adapters.<n>Our approach produces temporally coherent animations with minimal artifacts such as colour bleeding or object deformation.
• Score: 7.2542954248246305
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The production of high-quality 2D animation is highly labor-intensive process, as animators are currently required to draw and color a large number of frames by hand. We present SketchColour, the first sketch-to-colour pipeline for 2D animation built on a diffusion transformer (DiT) backbone. By replacing the conventional U-Net denoiser with a DiT-style architecture and injecting sketch information via lightweight channel-concatenation adapters accompanied with LoRA finetuning, our method natively integrates conditioning without the parameter and memory bloat of a duplicated ControlNet, greatly reducing parameter count and GPU memory usage. Evaluated on the SAKUGA dataset, SketchColour outperforms previous state-of-the-art video colourization methods across all metrics, despite using only half the training data of competing models. Our approach produces temporally coherent animations with minimal artifacts such as colour bleeding or object deformation. Our code is available at: https://bconstantine.github.io/SketchColour .

Related papers

• AnimeColor: Reference-based Animation Colorization with Diffusion Transformers [9.64847784171945]
  Animation colorization plays a vital role in animation production, yet existing methods struggle to achieve color accuracy and temporal consistency.<n>We propose textbfAnimeColor, a novel reference-based animation colorization framework leveraging Diffusion Transformers (DiT)<n>Our approach integrates sketch sequences into a DiT-based video diffusion model, enabling sketch-controlled animation generation.
  arXiv  Detail & Related papers  (2025-07-27T07:25:08Z)
• MagicColor: Multi-Instance Sketch Colorization [44.72374445094054]
  MagicColor is a diffusion-based framework for multi-instance sketch colorization.<n>Our model critically automates the colorization process with zero manual adjustments.
  arXiv  Detail & Related papers  (2025-03-21T08:53:14Z)
• AniDoc: Animation Creation Made Easier [54.97341104616779]
  Our research focuses on reducing the labor costs in the production of 2D animation by harnessing the potential of increasingly powerful AI.<n>AniDoc emerges as a video line art colorization tool, which automatically converts sketch sequences into colored animations.<n>Our model exploits correspondence matching as an explicit guidance, yielding strong robustness to the variations between the reference character and each line art frame.
  arXiv  Detail & Related papers  (2024-12-18T18:59:59Z)
• Paint Bucket Colorization Using Anime Character Color Design Sheets [72.66788521378864]
  We introduce inclusion matching, which allows the network to understand the relationships between segments. Our network's training pipeline significantly improves performance in both colorization and consecutive frame colorization. To support our network's training, we have developed a unique dataset named PaintBucket-Character.
  arXiv  Detail & Related papers  (2024-10-25T09:33:27Z)
• Learning Inclusion Matching for Animation Paint Bucket Colorization [76.4507878427755]
  We introduce a new learning-based inclusion matching pipeline, which directs the network to comprehend the inclusion relationships between segments. Our method features a two-stage pipeline that integrates a coarse color warping module with an inclusion matching module. To facilitate the training of our network, we also develope a unique dataset, referred to as PaintBucket-Character.
  arXiv  Detail & Related papers  (2024-03-27T08:32:48Z)
• Bridging the Gap: Sketch-Aware Interpolation Network for High-Quality Animation Sketch Inbetweening [58.09847349781176]
  We propose a novel deep learning method - Sketch-Aware Interpolation Network (SAIN) This approach incorporates multi-level guidance that formulates region-level correspondence, stroke-level correspondence and pixel-level dynamics. A multi-stream U-Transformer is then devised to characterize sketch inbetweening patterns using these multi-level guides through the integration of self / cross-attention mechanisms.
  arXiv  Detail & Related papers  (2023-08-25T09:51:03Z)
• Deep Animation Video Interpolation in the Wild [115.24454577119432]
  In this work, we formally define and study the animation video code problem for the first time. We propose an effective framework, AnimeInterp, with two dedicated modules in a coarse-to-fine manner. Notably, AnimeInterp shows favorable perceptual quality and robustness for animation scenarios in the wild.
  arXiv  Detail & Related papers  (2021-04-06T13:26:49Z)
• Self-Supervised Sketch-to-Image Synthesis [21.40315235087551]
  We study the exemplar-based sketch-to-image (s2i) synthesis task in a self-supervised learning manner. We first propose an unsupervised method to efficiently synthesize line-sketches for general RGB-only datasets. We then present a self-supervised Auto-Encoder (AE) to decouple the content/style features from sketches and RGB-images, and synthesize images that are both content-faithful to the sketches and style-consistent to the RGB-images.
  arXiv  Detail & Related papers  (2020-12-16T22:14:06Z)
• Multi-Density Sketch-to-Image Translation Network [65.4028451067947]
  We propose the first multi-level density sketch-to-image translation framework, which allows the input sketch to cover a wide range from rough object outlines to micro structures. Our method has been successfully verified on various datasets for different applications including face editing, multi-modal sketch-to-photo translation, and anime colorization.
  arXiv  Detail & Related papers  (2020-06-18T16:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.