Histogram-guided Video Colorization Structure with Spatial-Temporal Connection

• URL: http://arxiv.org/abs/2308.04899v1
• Date: Wed, 9 Aug 2023 11:59:18 GMT
• Title: Histogram-guided Video Colorization Structure with Spatial-Temporal Connection
• Authors: Zheyuan Liu, Pan Mu, Hanning Xu, Cong Bai
• Abstract summary: Histogram-guided Video Colorization with Spatial-Temporal connection structure (named ST-HVC) To fully exploit the chroma and motion information, the joint flow and histogram module is tailored to integrate the histogram and flow features. We show that the developed method achieves excellent performance both quantitatively and qualitatively in two video datasets.
• Score: 10.059070138875038
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video colorization, aiming at obtaining colorful and plausible results from grayish frames, has aroused a lot of interest recently. Nevertheless, how to maintain temporal consistency while keeping the quality of colorized results remains challenging. To tackle the above problems, we present a Histogram-guided Video Colorization with Spatial-Temporal connection structure (named ST-HVC). To fully exploit the chroma and motion information, the joint flow and histogram module is tailored to integrate the histogram and flow features. To manage the blurred and artifact, we design a combination scheme attending to temporal detail and flow feature combination. We further recombine the histogram, flow and sharpness features via a U-shape network. Extensive comparisons are conducted with several state-of-the-art image and video-based methods, demonstrating that the developed method achieves excellent performance both quantitatively and qualitatively in two video datasets.

Related papers

• LatentColorization: Latent Diffusion-Based Speaker Video Colorization [1.2641141743223379]
  We introduce a novel solution for achieving temporal consistency in video colorization. We demonstrate strong improvements on established image quality metrics compared to other existing methods. Our dataset encompasses a combination of conventional datasets and videos from television/movies.
  arXiv  Detail & Related papers  (2024-05-09T12:06:06Z)
• VideoComposer: Compositional Video Synthesis with Motion Controllability [52.4714732331632]
  VideoComposer allows users to flexibly compose a video with textual conditions, spatial conditions, and more importantly temporal conditions. We introduce the motion vector from compressed videos as an explicit control signal to provide guidance regarding temporal dynamics. In addition, we develop a Spatio-Temporal Condition encoder (STC-encoder) that serves as a unified interface to effectively incorporate the spatial and temporal relations of sequential inputs.
  arXiv  Detail & Related papers  (2023-06-03T06:29:02Z)
• Video Colorization with Pre-trained Text-to-Image Diffusion Models [19.807766482434563]
  We present ColorDiffuser, an adaptation of a pre-trained text-to-image latent diffusion model for video colorization. We propose two novel techniques to enhance the temporal coherence and maintain the vividness of colorization across frames.
  arXiv  Detail & Related papers  (2023-06-02T17:58:00Z)
• FlowChroma -- A Deep Recurrent Neural Network for Video Colorization [1.0499611180329804]
  We develop an automated video colorization framework that minimizes the flickering of colors across frames. We show that recurrent neural networks can be successfully used to improve color consistency in video colorization.
  arXiv  Detail & Related papers  (2023-05-23T05:41:53Z)
• Condensing a Sequence to One Informative Frame for Video Recognition [113.3056598548736]
  This paper studies a two-step alternative that first condenses the video sequence to an informative "frame" A valid question is how to define "useful information" and then distill from a sequence down to one synthetic frame. IFS consistently demonstrates evident improvements on image-based 2D networks and clip-based 3D networks.
  arXiv  Detail & Related papers  (2022-01-11T16:13:43Z)
• Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning [90.38674162878496]
  We propose a novel temporally consistent video colorization framework (TCVC) TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization. Experiments demonstrate that our method can not only obtain visually pleasing colorized video, but also achieve clearly better temporal consistency than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-09T13:00:14Z)
• Attention-guided Temporal Coherent Video Object Matting [78.82835351423383]
  We propose a novel deep learning-based object matting method that can achieve temporally coherent matting results. Its key component is an attention-based temporal aggregation module that maximizes image matting networks' strength. We show how to effectively solve the trimap generation problem by fine-tuning a state-of-the-art video object segmentation network.
  arXiv  Detail & Related papers  (2021-05-24T17:34:57Z)
• VCGAN: Video Colorization with Hybrid Generative Adversarial Network [22.45196398040388]
  Hybrid Video Colorization with Hybrid Generative Adversarative Network (VCGAN) is an improved approach to colorization using end-to-end learning. Experimental results demonstrate that VCGAN produces higher-quality and temporally more consistent colorful videos than existing approaches.
  arXiv  Detail & Related papers  (2021-04-26T05:50:53Z)
• Deep Video Matting via Spatio-Temporal Alignment and Aggregation [63.6870051909004]
  We propose a deep learning-based video matting framework which employs a novel aggregation feature module (STFAM) To eliminate frame-by-frame trimap annotations, a lightweight interactive trimap propagation network is also introduced. Our framework significantly outperforms conventional video matting and deep image matting methods.
  arXiv  Detail & Related papers  (2021-04-22T17:42:08Z)
• Progressive Temporal Feature Alignment Network for Video Inpainting [51.26380898255555]
  Video convolution aims to fill in-temporal "corrupted regions" with plausible content. Current methods achieve this goal through attention, flow-based warping, or 3D temporal convolution. We propose 'Progressive Temporal Feature Alignment Network', which progressively enriches features extracted from the current frame with the warped feature from neighbouring frames.
  arXiv  Detail & Related papers  (2021-04-08T04:50:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.