Prompt-based Consistent Video Colorization

• URL: http://arxiv.org/abs/2511.22330v1
• Date: Thu, 27 Nov 2025 11:01:06 GMT
• Title: Prompt-based Consistent Video Colorization
• Authors: Silvia Dani, Tiberio Uricchio, Lorenzo Seidenari,
• Abstract summary: We propose a novel approach to automate high-fidelity video colorization.<n>We employ a language-conditioned diffusion model to colorize grayscale frames.<n>We show our method achieves state-of-the-art performance in colorization accuracy (PSNR) and visual realism (Colorfulness, CDC)
• Score: 7.7741591842527455
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Existing video colorization methods struggle with temporal flickering or demand extensive manual input. We propose a novel approach automating high-fidelity video colorization using rich semantic guidance derived from language and segmentation. We employ a language-conditioned diffusion model to colorize grayscale frames. Guidance is provided via automatically generated object masks and textual prompts; our primary automatic method uses a generic prompt, achieving state-of-the-art results without specific color input. Temporal stability is achieved by warping color information from previous frames using optical flow (RAFT); a correction step detects and fixes inconsistencies introduced by warping. Evaluations on standard benchmarks (DAVIS30, VIDEVO20) show our method achieves state-of-the-art performance in colorization accuracy (PSNR) and visual realism (Colorfulness, CDC), demonstrating the efficacy of automated prompt-based guidance for consistent video colorization.

Related papers

• Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models [53.73253164099701]
  We introduce ColorWave, a training-free approach that achieves exact RGB-level color control in diffusion models without fine-tuning.<n>We demonstrate that ColorWave establishes a new paradigm for structured, color-consistent diffusion-based image synthesis.
  arXiv  Detail & Related papers  (2025-03-12T21:49:52Z)
• DreamColour: Controllable Video Colour Editing without Training [80.90808879991182]
  We present a training-free framework that makes precise video colour editing accessible through an intuitive interface.<n>By decoupling spatial and temporal aspects of colour editing, we can better align with users' natural workflow.<n>Our approach matches or exceeds state-of-the-art methods while eliminating the need for training or specialized hardware.
  arXiv  Detail & Related papers  (2024-12-06T16:57:54Z)
• L-C4: Language-Based Video Colorization for Creative and Consistent Color [59.069498113050436]
  We present Language-based video colorization for Creative and Consistent Colors (L-C4) Our model is built upon a pre-trained cross-modality generative model. We propose temporally deformable attention to prevent flickering or color shifts, and cross-clip fusion to maintain long-term color consistency.
  arXiv  Detail & Related papers  (2024-10-07T12:16:21Z)
• LatentColorization: Latent Diffusion-Based Speaker Video Colorization [1.2641141743223379]
  We introduce a novel solution for achieving temporal consistency in video colorization. We demonstrate strong improvements on established image quality metrics compared to other existing methods. Our dataset encompasses a combination of conventional datasets and videos from television/movies.
  arXiv  Detail & Related papers  (2024-05-09T12:06:06Z)
• Diffusing Colors: Image Colorization with Text Guided Diffusion [11.727899027933466]
  We present a novel image colorization framework that utilizes image diffusion techniques with granular text prompts. Our method provides a balance between automation and control, outperforming existing techniques in terms of visual quality and semantic coherence. Our approach holds potential particularly for color enhancement and historical image colorization.
  arXiv  Detail & Related papers  (2023-12-07T08:59:20Z)
• Improving Video Colorization by Test-Time Tuning [79.67548221384202]
  We propose an effective method, which aims to enhance video colorization through test-time tuning. By exploiting the reference to construct additional training samples during testing, our approach achieves a performance boost of 13 dB in PSNR on average.
  arXiv  Detail & Related papers  (2023-06-25T05:36:40Z)
• Video Colorization with Pre-trained Text-to-Image Diffusion Models [19.807766482434563]
  We present ColorDiffuser, an adaptation of a pre-trained text-to-image latent diffusion model for video colorization. We propose two novel techniques to enhance the temporal coherence and maintain the vividness of colorization across frames.
  arXiv  Detail & Related papers  (2023-06-02T17:58:00Z)
• L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors [62.80068955192816]
  We propose a unified model to perform language-based colorization with any-level descriptions. We leverage the pretrained cross-modality generative model for its robust language understanding and rich color priors. With the proposed novel sampling strategy, our model achieves instance-aware colorization in diverse and complex scenarios.
  arXiv  Detail & Related papers  (2023-05-24T14:57:42Z)
• Temporal Consistent Automatic Video Colorization via Semantic Correspondence [12.107878178519128]
  We propose a novel video colorization framework, which combines semantic correspondence into automatic video colorization. In the NTIRE 2023 Video Colorization Challenge, our method ranks at the 3rd place in Color Distribution Consistency (CDC) Optimization track.
  arXiv  Detail & Related papers  (2023-05-13T12:06:09Z)
• Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning [90.38674162878496]
  We propose a novel temporally consistent video colorization framework (TCVC) TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization. Experiments demonstrate that our method can not only obtain visually pleasing colorized video, but also achieve clearly better temporal consistency than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-09T13:00:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.