DreamColour: Controllable Video Colour Editing without Training

• URL: http://arxiv.org/abs/2412.05180v1
• Date: Fri, 06 Dec 2024 16:57:54 GMT
• Title: DreamColour: Controllable Video Colour Editing without Training
• Authors: Chaitat Utintu, Pinaki Nath Chowdhury, Aneeshan Sain, Subhadeep Koley, Ayan Kumar Bhunia, Yi-Zhe Song,
• Abstract summary: We present a training-free framework that makes precise video colour editing accessible through an intuitive interface. By decoupling spatial and temporal aspects of colour editing, we can better align with users' natural workflow. Our approach matches or exceeds state-of-the-art methods while eliminating the need for training or specialized hardware.
• Score: 80.90808879991182
• License:
• Abstract: Video colour editing is a crucial task for content creation, yet existing solutions either require painstaking frame-by-frame manipulation or produce unrealistic results with temporal artefacts. We present a practical, training-free framework that makes precise video colour editing accessible through an intuitive interface while maintaining professional-quality output. Our key insight is that by decoupling spatial and temporal aspects of colour editing, we can better align with users' natural workflow -- allowing them to focus on precise colour selection in key frames before automatically propagating changes across time. We achieve this through a novel technical framework that combines: (i) a simple point-and-click interface merging grid-based colour selection with automatic instance segmentation for precise spatial control, (ii) bidirectional colour propagation that leverages inherent video motion patterns, and (iii) motion-aware blending that ensures smooth transitions even with complex object movements. Through extensive evaluation on diverse scenarios, we demonstrate that our approach matches or exceeds state-of-the-art methods while eliminating the need for training or specialized hardware, making professional-quality video colour editing accessible to everyone.

Related papers

• FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors [64.54220123913154]
  We introduce FramePainter as an efficient instantiation of image-to-video generation problem. It only uses a lightweight sparse control encoder to inject editing signals. It domainantly outperforms previous state-of-the-art methods with far less training data.
  arXiv  Detail & Related papers  (2025-01-14T16:09:16Z)
• UniPaint: Unified Space-time Video Inpainting via Mixture-of-Experts [20.955898491009656]
  UniPaint is a generative space-time video inpainting framework that enables spatial-temporal inpainting. UniPaint produces high-quality and aesthetically pleasing results, achieving the best results across various tasks and scale setups.
  arXiv  Detail & Related papers  (2024-12-09T09:45:14Z)
• L-C4: Language-Based Video Colorization for Creative and Consistent Color [59.069498113050436]
  We present Language-based video colorization for Creative and Consistent Colors (L-C4) Our model is built upon a pre-trained cross-modality generative model. We propose temporally deformable attention to prevent flickering or color shifts, and cross-clip fusion to maintain long-term color consistency.
  arXiv  Detail & Related papers  (2024-10-07T12:16:21Z)
• LatentColorization: Latent Diffusion-Based Speaker Video Colorization [1.2641141743223379]
  We introduce a novel solution for achieving temporal consistency in video colorization. We demonstrate strong improvements on established image quality metrics compared to other existing methods. Our dataset encompasses a combination of conventional datasets and videos from television/movies.
  arXiv  Detail & Related papers  (2024-05-09T12:06:06Z)
• Learning Inclusion Matching for Animation Paint Bucket Colorization [76.4507878427755]
  We introduce a new learning-based inclusion matching pipeline, which directs the network to comprehend the inclusion relationships between segments. Our method features a two-stage pipeline that integrates a coarse color warping module with an inclusion matching module. To facilitate the training of our network, we also develope a unique dataset, referred to as PaintBucket-Character.
  arXiv  Detail & Related papers  (2024-03-27T08:32:48Z)
• FlowChroma -- A Deep Recurrent Neural Network for Video Colorization [1.0499611180329804]
  We develop an automated video colorization framework that minimizes the flickering of colors across frames. We show that recurrent neural networks can be successfully used to improve color consistency in video colorization.
  arXiv  Detail & Related papers  (2023-05-23T05:41:53Z)
• Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning [90.38674162878496]
  We propose a novel temporally consistent video colorization framework (TCVC) TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization. Experiments demonstrate that our method can not only obtain visually pleasing colorized video, but also achieve clearly better temporal consistency than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-09T13:00:14Z)
• Task-agnostic Temporally Consistent Facial Video Editing [84.62351915301795]
  We propose a task-agnostic, temporally consistent facial video editing framework. Based on a 3D reconstruction model, our framework is designed to handle several editing tasks in a more unified and disentangled manner. Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.
  arXiv  Detail & Related papers  (2020-07-03T02:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.