Related papers: Towards Automated Movie Trailer Generation

Towards Automated Movie Trailer Generation

URL: http://arxiv.org/abs/2404.03477v1
Date: Thu, 4 Apr 2024 14:28:34 GMT
Title: Towards Automated Movie Trailer Generation
Authors: Dawit Mureja Argaw, Mattia Soldan, Alejandro Pardo, Chen Zhao, Fabian Caba Heilbron, Joon Son Chung, Bernard Ghanem,
Abstract summary: We introduce Trailer Generation Transformer (TGT), a deep-learning framework utilizing an encoder-decoder architecture. TGT movie encoder is tasked with contextualizing each movie shot representation via self-attention, while the autoregressive trailer decoder predicts the feature representation of the next trailer shot. Our TGT significantly outperforms previous methods on a comprehensive suite of metrics.
Score: 98.9854474456265
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Movie trailers are an essential tool for promoting films and attracting audiences. However, the process of creating trailers can be time-consuming and expensive. To streamline this process, we propose an automatic trailer generation framework that generates plausible trailers from a full movie by automating shot selection and composition. Our approach draws inspiration from machine translation techniques and models the movies and trailers as sequences of shots, thus formulating the trailer generation problem as a sequence-to-sequence task. We introduce Trailer Generation Transformer (TGT), a deep-learning framework utilizing an encoder-decoder architecture. TGT movie encoder is tasked with contextualizing each movie shot representation via self-attention, while the autoregressive trailer decoder predicts the feature representation of the next trailer shot, accounting for the relevance of shots' temporal order in trailers. Our TGT significantly outperforms previous methods on a comprehensive suite of metrics.

Related papers

ScreenWriter: Automatic Screenplay Generation and Movie Summarisation [55.20132267309382]
Video content has driven demand for textual descriptions or summaries that allow users to recall key plot points or get an overview without watching. We propose the task of automatic screenplay generation, and a method, ScreenWriter, that operates only on video and produces output which includes dialogue, speaker names, scene breaks, and visual descriptions. ScreenWriter introduces a novel algorithm to segment the video into scenes based on the sequence of visual vectors, and a novel method for the challenging problem of determining character names, based on a database of actors' faces.
arXiv Detail & Related papers (2024-10-17T07:59:54Z)
MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequence [62.72540590546812]
MovieDreamer is a novel hierarchical framework that integrates the strengths of autoregressive models with diffusion-based rendering. We present experiments across various movie genres, demonstrating that our approach achieves superior visual and narrative quality.
arXiv Detail & Related papers (2024-07-23T17:17:05Z)
Movie101v2: Improved Movie Narration Benchmark [53.54176725112229]
Automatic movie narration aims to generate video-aligned plot descriptions to assist visually impaired audiences. We introduce Movie101v2, a large-scale, bilingual dataset with enhanced data quality specifically designed for movie narration. Based on our new benchmark, we baseline a range of large vision-language models, including GPT-4V, and conduct an in-depth analysis of the challenges in narration generation.
arXiv Detail & Related papers (2024-04-20T13:15:27Z)
Find the Cliffhanger: Multi-Modal Trailerness in Soap Operas [17.476344577463525]
We introduce a multi-modal method for predicting the trailerness to assist editors in selecting trailer- worthy moments from long-form videos. We present results on a newly introduced soap opera dataset, demonstrating that predicting trailerness is a challenging task.
arXiv Detail & Related papers (2024-01-29T11:34:36Z)
AI based approach to Trailer Generation for Online Educational Courses [0.0]
The framework we propose is a template based method for video trailer generation. The proposed trailer is in the form of a timeline consisting of various fragments created by selecting, para-phrasing or generating content. We perform user evaluation with 63 human evaluators for evaluating the trailers generated by our system.
arXiv Detail & Related papers (2023-01-10T13:33:08Z)
Film Trailer Generation via Task Decomposition [65.16768855902268]
We model movies as graphs, where nodes are shots and edges denote semantic relations between them. We learn these relations using joint contrastive training which leverages privileged textual information from screenplays. An unsupervised algorithm then traverses the graph and generates trailers that human judges prefer to ones generated by competitive supervised approaches.
arXiv Detail & Related papers (2021-11-16T20:50:52Z)
A Case Study of Deep Learning Based Multi-Modal Methods for Predicting the Age-Suitability Rating of Movie Trailers [15.889598494755646]
We introduce a new dataset containing videos of movie trailers in English downloaded from IMDB and YouTube. We propose a multi-modal deep learning pipeline addressing the movie trailer age suitability rating problem.
arXiv Detail & Related papers (2021-01-26T17:15:35Z)
Learning Trailer Moments in Full-Length Movies [49.74693903050302]
We leverage the officially-released trailers as the weak supervision to learn a model that can detect the key moments from full-length movies. We introduce a novel ranking network that utilizes the Co-Attention between movies and trailers as guidance to generate the training pairs. We construct the first movie-trailer dataset, and the proposed Co-Attention assisted ranking network shows superior performance even over the supervised approach.
arXiv Detail & Related papers (2020-08-19T15:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.