StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description

• URL: http://arxiv.org/abs/2607.11798v1
• Date: Mon, 13 Jul 2026 16:50:03 GMT
• Title: StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description
• Abstract summary: We propose a training-free framework for story-aware long-form audio descriptions.<n>StoryTeller maintains a verified narrative memory that carries forward story-relevant information across scenes.
• Score: 2.26752781534698
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-form audio description (AD) requires more than describing visible actions: it must preserve characters, events, relationships, and story context across scenes so that blind and low-vision (BLV) audiences can follow a film. Modern video-language models (VLMs) are effective on short clips, but they often treat each moment independently, producing descriptions that miss who characters are, why events matter, and how the current scene connects to earlier narrative context. We propose StoryTeller, a training-free framework for story-aware long-form AD. Instead of relying only on local visual cues, StoryTeller maintains a verified narrative memory that carries forward story-relevant information across scenes, enabling later descriptions to remain coherent, grounded, and contextually informative. Given only raw video and a movie title, StoryTeller can optionally retrieve public movie metadata to resolve names and story context, while accepting only facts that are supported by the video through semantic filtering and VLM verification. The method requires no subtitles, scripts, AD transcripts, aligned captions, character banks, precomputed face identities, or task-specific fine-tuning. To evaluate whether generated AD preserves narrative information, we introduce StoryAD-QA, a question-answering benchmark that tests whether a language model can answer story-context questions using only the generated descriptions. Experiments on standard AD benchmarks and diverse long-form videos show that StoryTeller consistently improves narrative coherence, factual grounding, and story comprehension over strong baselines in automatic, QA-based, and human evaluations.

Related papers

• REFRAMED: Towards Realistic Audio Description Generation for Movies [53.11641004249926]
  Audio Description (AD) is a verbal narration of key visual content in videos, enabling access for visually impaired audiences.<n>Existing approaches formulate AD generation in an artificial setting where both the content and timing of descriptions are pre-specified.<n>We introduce a new formulation of AD generation in which models must jointly decide what to describe and when to do it.
  arXiv  Detail & Related papers  (2026-08-10T15:55:56Z)
• STAGE: A Benchmark for Knowledge Graph Construction, Question Answering, and In-Script Role-Playing over Movie Screenplays [16.069095458601588]
  We introduce STAGE (Screenplay Text, Agents, Graphs and Evaluation), a unified benchmark for narrative understanding over full-length movie screenplays.<n> STAGE defines four tasks: knowledge graph construction, scene-level event summarization, long-context screenplay question answering, and in-script character role-playing, all grounded in a shared narrative world representation.<n>The benchmark provides cleaned scripts, curated knowledge graphs, and event- and character-centric annotations for 150 films across English and Chinese, enabling holistic evaluation of models' abilities to build world representations, abstract and verify narrative events, reason over long narratives, and generate
  arXiv  Detail & Related papers  (2026-01-13T12:50:58Z)
• ScreenWriter: Automatic Screenplay Generation and Movie Summarisation [55.20132267309382]
  Video content has driven demand for textual descriptions or summaries that allow users to recall key plot points or get an overview without watching. We propose the task of automatic screenplay generation, and a method, ScreenWriter, that operates only on video and produces output which includes dialogue, speaker names, scene breaks, and visual descriptions. ScreenWriter introduces a novel algorithm to segment the video into scenes based on the sequence of visual vectors, and a novel method for the challenging problem of determining character names, based on a database of actors' faces.
  arXiv  Detail & Related papers  (2024-10-17T07:59:54Z)
• Select and Summarize: Scene Saliency for Movie Script Summarization [11.318175666743656]
  We introduce a scene saliency dataset that consists of human-annotated salient scenes for 100 movies. We propose a two-stage abstractive summarization approach which first identifies the salient scenes in script and then generates a summary using only those scenes.
  arXiv  Detail & Related papers  (2024-04-04T16:16:53Z)
• Long Story Short: a Summarize-then-Search Method for Long Video Question Answering [23.094728230459125]
  We investigate if language models can extend their zero-shot reasoning abilities to long multimodal narratives in multimedia content. We propose Long Story Short, a framework for narrative video QA that first summarizes the narrative of the video to a short plot and then searches parts of the video relevant to the question. Our model outperforms state-of-the-art supervised models by a large margin, highlighting the potential of zero-shot QA for long videos.
  arXiv  Detail & Related papers  (2023-11-02T13:36:11Z)
• HowToCaption: Prompting LLMs to Transform Video Annotations at Scale [72.69268311756082]
  We propose to leverage the capabilities of large language models (LLMs) to obtain high-quality video descriptions aligned with videos at scale. We introduce a prompting method that is able to take into account a longer text of subtitles, allowing us to capture the contextual information beyond one single sentence. We apply our method to the subtitles of the HowTo100M dataset, creating a new large-scale dataset, HowToCaption.
  arXiv  Detail & Related papers  (2023-10-07T19:32:55Z)
• Album Storytelling with Iterative Story-aware Captioning and Large Language Models [86.6548090965982]
  We study how to transform an album to vivid and coherent stories, a task we refer to as "album storytelling" With recent advances in Large Language Models (LLMs), it is now possible to generate lengthy, coherent text. Our method effectively generates more accurate and engaging stories for albums, with enhanced coherence and vividness.
  arXiv  Detail & Related papers  (2023-05-22T11:45:10Z)
• Movie101: A New Movie Understanding Benchmark [47.24519006577205]
  We construct a large-scale Chinese movie benchmark, named Movie101. We propose a new metric called Movie Narration Score (MNScore) for movie narrating evaluation. For both two tasks, our proposed methods well leverage external knowledge and outperform carefully designed baselines.
  arXiv  Detail & Related papers  (2023-05-20T08:43:51Z)
• HierVL: Learning Hierarchical Video-Language Embeddings [108.77600799637172]
  HierVL is a novel hierarchical video-language embedding that simultaneously accounts for both long-term and short-term associations. We introduce a hierarchical contrastive training objective that encourages text-visual alignment at both the clip level and video level. Our hierarchical scheme yields a clip representation that outperforms its single-level counterpart as well as a long-term video representation that achieves SotA.
  arXiv  Detail & Related papers  (2023-01-05T21:53:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.