Consistent Video Colorization via Palette Guidance

• URL: http://arxiv.org/abs/2501.19331v1
• Date: Fri, 31 Jan 2025 17:31:19 GMT
• Title: Consistent Video Colorization via Palette Guidance
• Authors: Han Wang, Yuang Zhang, Yuhong Zhang, Lingxiao Lu, Li Song,
• Abstract summary: We regard the colorization task as a generative task and introduce Stable Video Diffusion (SVD) as our base model. We design a palette-based color guider to assist the model in generating vivid and consistent colors. Experiments demonstrate that the proposed method can provide vivid and stable colors for videos, surpassing previous methods.
• Score: 10.651227296134655
• License:
• Abstract: Colorization is a traditional computer vision task and it plays an important role in many time-consuming tasks, such as old film restoration. Existing methods suffer from unsaturated color and temporally inconsistency. In this paper, we propose a novel pipeline to overcome the challenges. We regard the colorization task as a generative task and introduce Stable Video Diffusion (SVD) as our base model. We design a palette-based color guider to assist the model in generating vivid and consistent colors. The color context introduced by the palette not only provides guidance for color generation, but also enhances the stability of the generated colors through a unified color context across multiple sequences. Experiments demonstrate that the proposed method can provide vivid and stable colors for videos, surpassing previous methods.

Related papers

• VanGogh: A Unified Multimodal Diffusion-based Framework for Video Colorization [53.35016574938809]
  Video colorization aims to transform grayscale videos into vivid color representations while maintaining temporal consistency and structural integrity. Existing video colorization methods often suffer from color bleeding and lack comprehensive control. We introduce VanGogh, a unified multimodal diffusion-based framework for video colorization.
  arXiv  Detail & Related papers  (2025-01-16T12:20:40Z)
• ColorFlow: Retrieval-Augmented Image Sequence Colorization [65.93834649502898]
  We propose a three-stage diffusion-based framework tailored for image sequence colorization in industrial applications. Unlike existing methods that require per-ID finetuning or explicit ID embedding extraction, we propose a novel Retrieval Augmented Colorization pipeline. Our pipeline also features a dual-branch design: one branch for color identity extraction and the other for colorization.
  arXiv  Detail & Related papers  (2024-12-16T14:32:49Z)
• Paint Bucket Colorization Using Anime Character Color Design Sheets [72.66788521378864]
  We introduce inclusion matching, which allows the network to understand the relationships between segments. Our network's training pipeline significantly improves performance in both colorization and consecutive frame colorization. To support our network's training, we have developed a unique dataset named PaintBucket-Character.
  arXiv  Detail & Related papers  (2024-10-25T09:33:27Z)
• LatentColorization: Latent Diffusion-Based Speaker Video Colorization [1.2641141743223379]
  We introduce a novel solution for achieving temporal consistency in video colorization. We demonstrate strong improvements on established image quality metrics compared to other existing methods. Our dataset encompasses a combination of conventional datasets and videos from television/movies.
  arXiv  Detail & Related papers  (2024-05-09T12:06:06Z)
• Control Color: Multimodal Diffusion-based Interactive Image Colorization [81.68817300796644]
  Control Color (Ctrl Color) is a multi-modal colorization method that leverages the pre-trained Stable Diffusion (SD) model. We present an effective way to encode user strokes to enable precise local color manipulation. We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
  arXiv  Detail & Related papers  (2024-02-16T17:51:13Z)
• Diffusing Colors: Image Colorization with Text Guided Diffusion [11.727899027933466]
  We present a novel image colorization framework that utilizes image diffusion techniques with granular text prompts. Our method provides a balance between automation and control, outperforming existing techniques in terms of visual quality and semantic coherence. Our approach holds potential particularly for color enhancement and historical image colorization.
  arXiv  Detail & Related papers  (2023-12-07T08:59:20Z)
• Video Colorization with Pre-trained Text-to-Image Diffusion Models [19.807766482434563]
  We present ColorDiffuser, an adaptation of a pre-trained text-to-image latent diffusion model for video colorization. We propose two novel techniques to enhance the temporal coherence and maintain the vividness of colorization across frames.
  arXiv  Detail & Related papers  (2023-06-02T17:58:00Z)
• PalGAN: Image Colorization with Palette Generative Adversarial Networks [51.59276436217957]
  We propose a new GAN-based colorization approach PalGAN, integrated with palette estimation and chromatic attention. PalGAN outperforms state-of-the-arts in quantitative evaluation and visual comparison, delivering notable diverse, contrastive, and edge-preserving appearances.
  arXiv  Detail & Related papers  (2022-10-20T12:28:31Z)
• Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning [90.38674162878496]
  We propose a novel temporally consistent video colorization framework (TCVC) TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization. Experiments demonstrate that our method can not only obtain visually pleasing colorized video, but also achieve clearly better temporal consistency than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-09T13:00:14Z)
• Towards Vivid and Diverse Image Colorization with Generative Color Prior [17.087464490162073]
  Recent deep-learning-based methods could automatically colorize images at a low cost. We aim at recovering vivid colors by leveraging the rich and diverse color priors encapsulated in a pretrained Generative Adversarial Networks (GAN) Thanks to the powerful generative color prior and delicate designs, our method could produce vivid colors with a single forward pass.
  arXiv  Detail & Related papers  (2021-08-19T17:49:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.