Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition

• URL: http://arxiv.org/abs/2111.08774v2
• Date: Mon, 30 Dec 2024 13:43:54 GMT
• Title: Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition
• Authors: Pinelopi Papalampidi, Frank Keller, Mirella Lapata,
• Abstract summary: We focus on finding trailer moments in a movie, i.e., shots that could be potentially included in a trailer. We model movies as graphs, where nodes are shots and edges denote semantic relations between them. An unsupervised algorithm then traverses the graph and selects trailer moments from the movie that human judges prefer to ones selected by competitive supervised approaches. Our tool allows users to select trailer shots in under 30 minutes that are superior to fully automatic methods and comparable to (exclusive) manual selection by experts.
• Score: 63.842627949509414
• License:
• Abstract: Movie trailers perform multiple functions: they introduce viewers to the story, convey the mood and artistic style of the film, and encourage audiences to see the movie. These diverse functions make trailer creation a challenging endeavor. In this work, we focus on finding trailer moments in a movie, i.e., shots that could be potentially included in a trailer. We decompose this task into two subtasks: narrative structure identification and sentiment prediction. We model movies as graphs, where nodes are shots and edges denote semantic relations between them. We learn these relations using joint contrastive training which distills rich textual information (e.g., characters, actions, situations) from screenplays. An unsupervised algorithm then traverses the graph and selects trailer moments from the movie that human judges prefer to ones selected by competitive supervised approaches. A main advantage of our algorithm is that it uses interpretable criteria, which allows us to deploy it in an interactive tool for trailer creation with a human in the loop. Our tool allows users to select trailer shots in under 30 minutes that are superior to fully automatic methods and comparable to (exclusive) manual selection by experts.

Related papers

• ScreenWriter: Automatic Screenplay Generation and Movie Summarisation [55.20132267309382]
  Video content has driven demand for textual descriptions or summaries that allow users to recall key plot points or get an overview without watching. We propose the task of automatic screenplay generation, and a method, ScreenWriter, that operates only on video and produces output which includes dialogue, speaker names, scene breaks, and visual descriptions. ScreenWriter introduces a novel algorithm to segment the video into scenes based on the sequence of visual vectors, and a novel method for the challenging problem of determining character names, based on a database of actors' faces.
  arXiv  Detail & Related papers  (2024-10-17T07:59:54Z)
• Towards Automated Movie Trailer Generation [98.9854474456265]
  We introduce Trailer Generation Transformer (TGT), a deep-learning framework utilizing an encoder-decoder architecture. TGT movie encoder is tasked with contextualizing each movie shot representation via self-attention, while the autoregressive trailer decoder predicts the feature representation of the next trailer shot. Our TGT significantly outperforms previous methods on a comprehensive suite of metrics.
  arXiv  Detail & Related papers  (2024-04-04T14:28:34Z)
• Find the Cliffhanger: Multi-Modal Trailerness in Soap Operas [17.476344577463525]
  We introduce a multi-modal method for predicting the trailerness to assist editors in selecting trailer- worthy moments from long-form videos. We present results on a newly introduced soap opera dataset, demonstrating that predicting trailerness is a challenging task.
  arXiv  Detail & Related papers  (2024-01-29T11:34:36Z)
• AI based approach to Trailer Generation for Online Educational Courses [0.0]
  The framework we propose is a template based method for video trailer generation. The proposed trailer is in the form of a timeline consisting of various fragments created by selecting, para-phrasing or generating content. We perform user evaluation with 63 human evaluators for evaluating the trailers generated by our system.
  arXiv  Detail & Related papers  (2023-01-10T13:33:08Z)
• Multilevel profiling of situation and dialogue-based deep networks for movie genre classification using movie trailers [7.904790547594697]
  We propose a novel multi-modality: situation, dialogue, and metadata-based movie genre classification framework. We develop the English movie trailer dataset (EMTD), which contains 2000 Hollywood movie trailers belonging to five popular genres.
  arXiv  Detail & Related papers  (2021-09-14T07:33:56Z)
• Where to look at the movies : Analyzing visual attention to understand movie editing [75.16856363008128]
  We propose a new eye-tracking database, containing gaze pattern information on movie sequences. We show how state-of-the-art computational saliency techniques behave on this dataset.
  arXiv  Detail & Related papers  (2021-02-26T09:54:58Z)
• Movie Summarization via Sparse Graph Construction [65.16768855902268]
  We propose a model that identifies TP scenes by building a sparse movie graph that represents relations between scenes and is constructed using multimodal information. According to human judges, the summaries created by our approach are more informative and complete, and receive higher ratings, than the outputs of sequence-based models and general-purpose summarization algorithms.
  arXiv  Detail & Related papers  (2020-12-14T13:54:34Z)
• Learning Trailer Moments in Full-Length Movies [49.74693903050302]
  We leverage the officially-released trailers as the weak supervision to learn a model that can detect the key moments from full-length movies. We introduce a novel ranking network that utilizes the Co-Attention between movies and trailers as guidance to generate the training pairs. We construct the first movie-trailer dataset, and the proposed Co-Attention assisted ranking network shows superior performance even over the supervised approach.
  arXiv  Detail & Related papers  (2020-08-19T15:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.