A Survey on Video Diffusion Models

• URL: http://arxiv.org/abs/2310.10647v2
• Date: Sat, 14 Sep 2024 08:38:20 GMT
• Title: A Survey on Video Diffusion Models
• Authors: Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, Yu-Gang Jiang,
• Abstract summary: The recent wave of AI-generated content (AIGC) has witnessed substantial success in computer vision. Due to their impressive generative capabilities, diffusion models are gradually superseding methods based on GANs and auto-regressive Transformers. This paper presents a comprehensive review of video diffusion models in the AIGC era.
• Score: 103.03565844371711
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent wave of AI-generated content (AIGC) has witnessed substantial success in computer vision, with the diffusion model playing a crucial role in this achievement. Due to their impressive generative capabilities, diffusion models are gradually superseding methods based on GANs and auto-regressive Transformers, demonstrating exceptional performance not only in image generation and editing, but also in the realm of video-related research. However, existing surveys mainly focus on diffusion models in the context of image generation, with few up-to-date reviews on their application in the video domain. To address this gap, this paper presents a comprehensive review of video diffusion models in the AIGC era. Specifically, we begin with a concise introduction to the fundamentals and evolution of diffusion models. Subsequently, we present an overview of research on diffusion models in the video domain, categorizing the work into three key areas: video generation, video editing, and other video understanding tasks. We conduct a thorough review of the literature in these three key areas, including further categorization and practical contributions in the field. Finally, we discuss the challenges faced by research in this domain and outline potential future developmental trends. A comprehensive list of video diffusion models studied in this survey is available at https://github.com/ChenHsing/Awesome-Video-Diffusion-Models.

Related papers

• Diffusion Model-Based Video Editing: A Survey [47.45047496559506]
  This paper reviews diffusion model-based video editing techniques, including theoretical foundations and practical applications. We categorize video editing approaches by the inherent connections of their core technologies, depicting evolutionary trajectory. This paper also dives into novel applications, including point-based editing and pose-guided human video editing.
  arXiv  Detail & Related papers  (2024-06-26T04:58:39Z)
• Diffusion Models in Low-Level Vision: A Survey [82.77962165415153]
  diffusion model-based solutions have emerged as widely acclaimed for their ability to produce samples of superior quality and diversity. We present three generic diffusion modeling frameworks and explore their correlations with other deep generative models. We summarize extended diffusion models applied in other tasks, including medical, remote sensing, and video scenarios.
  arXiv  Detail & Related papers  (2024-06-17T01:49:27Z)
• Video Diffusion Models: A Survey [3.7985353171858045]
  Diffusion generative models have recently become a robust technique for producing and modifying coherent, high-quality video. This survey offers a systematic overview of critical elements of diffusion models for video generation, covering applications, architectural choices, and the modeling of temporal dynamics.
  arXiv  Detail & Related papers  (2024-05-06T04:01:42Z)
• State of the Art on Diffusion Models for Visual Computing [191.6168813012954]
  This report introduces the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model. We also give a comprehensive overview of the rapidly growing literature on diffusion-based generation and editing. We discuss available datasets, metrics, open challenges, and social implications.
  arXiv  Detail & Related papers  (2023-10-11T05:32:29Z)
• Diffusion Models for Time Series Applications: A Survey [23.003273147019446]
  Diffusion models are used in image, video, and text synthesis nowadays. We focus on diffusion-based methods for time series forecasting, imputation, and generation. We conclude the common limitation of diffusion-based methods and highlight potential future research directions.
  arXiv  Detail & Related papers  (2023-05-01T02:06:46Z)
• Diffusion Models in Vision: A Survey [80.82832715884597]
  A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
  arXiv  Detail & Related papers  (2022-09-10T22:00:30Z)
• A Survey on Generative Diffusion Model [75.93774014861978]
  Diffusion models are an emerging class of deep generative models. They have certain limitations, including a time-consuming iterative generation process and confinement to high-dimensional Euclidean space. This survey presents a plethora of advanced techniques aimed at enhancing diffusion models.
  arXiv  Detail & Related papers  (2022-09-06T16:56:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.