A Text-Native Interface for Generative Video Authoring

• URL: http://arxiv.org/abs/2603.09072v1
• Date: Tue, 10 Mar 2026 01:33:41 GMT
• Title: A Text-Native Interface for Generative Video Authoring
• Abstract summary: We introduce Doki, a text-native interface for generative video authoring.<n>In Doki, writing text is the primary interaction: within a single document, users define assets, structure scenes, create shots, refine edits, and add audio.<n>This work contributes a fundamental shift in generative video interfaces, demonstrating a powerful and accessible new way to craft visual stories.
• Score: 5.640604615309813
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Everyone can write their stories in freeform text format -- it's something we all learn in school. Yet storytelling via video requires one to learn specialized and complicated tools. In this paper, we introduce Doki, a text-native interface for generative video authoring, aligning video creation with the natural process of text writing. In Doki, writing text is the primary interaction: within a single document, users define assets, structure scenes, create shots, refine edits, and add audio. We articulate the design principles of this text-first approach and demonstrate Doki's capabilities through a series of examples. To evaluate its real-world use, we conducted a week-long deployment study with participants of varying expertise in video authoring. This work contributes a fundamental shift in generative video interfaces, demonstrating a powerful and accessible new way to craft visual stories.

Related papers

• EagleNet: Energy-Aware Fine-Grained Relationship Learning Network for Text-Video Retrieval [59.858268313337355]
  We introduce Energy-Aware Fine-Grained Relationship Learning Network (EagleNet)<n>EagleNet generates accurate and context-aware enriched text embeddings.<n>Experiments have demonstrated the superiority of EagleNet across MSRVTT, DiDeMo, MSVD, and VATEX.
  arXiv  Detail & Related papers  (2026-03-26T10:05:19Z)
• Rewriting Video: Text-Driven Reauthoring of Video Footage [19.217919638665713]
  Video is a powerful medium for communication and storytelling, yet reauthoring existing footage remains challenging.<n>Recent advances in generative AI suggest a new paradigm: what if editing a video were as straightforward as rewriting text?<n>We present a tech probe and a study on text-driven video reauthoring.
  arXiv  Detail & Related papers  (2026-01-13T13:49:05Z)
• Text-Animator: Controllable Visual Text Video Generation [149.940821790235]
  We propose an innovative approach termed Text-Animator for visual text video generation. Text-Animator contains a text embedding injection module to precisely depict the structures of visual text in generated videos. We also develop a camera control module and a text refinement module to improve the stability of generated visual text.
  arXiv  Detail & Related papers  (2024-06-25T17:59:41Z)
• CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects [74.61964363605632]
  Current approaches for personalizing text-to-video generation suffer from tackling multiple subjects.<n>We propose CustomVideo, a novel framework that can generate identity-preserving videos with the guidance of multiple subjects.
  arXiv  Detail & Related papers  (2024-01-18T13:23:51Z)
• Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation [69.20173154096]
  We develop a framework comprised of two functional modules, Motion Structure Retrieval and Structure-Guided Text-to-Video Synthesis. For the first module, we leverage an off-the-shelf video retrieval system and extract video depths as motion structure. For the second module, we propose a controllable video generation model that offers flexible controls over structure and characters.
  arXiv  Detail & Related papers  (2023-07-13T17:57:13Z)
• A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot [67.00455874279383]
  We propose verbalizing long videos to generate descriptions in natural language, then performing video-understanding tasks on the generated story as opposed to the original video. Our method, despite being zero-shot, achieves significantly better results than supervised baselines for video understanding. To alleviate a lack of story understanding benchmarks, we publicly release the first dataset on a crucial task in computational social science on persuasion strategy identification.
  arXiv  Detail & Related papers  (2023-05-16T19:13:11Z)
• FairyTailor: A Multimodal Generative Framework for Storytelling [33.39639788612019]
  We introduce a system and a demo, FairyTailor, for human-in-the-loop visual story co-creation. Users can create a cohesive children's fairytale by weaving generated texts and retrieved images with their input. To our knowledge, this is the first dynamic tool for multimodal story generation that allows interactive co-formation of both texts and images.
  arXiv  Detail & Related papers  (2021-07-13T02:45:08Z)
• VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs [103.99315770490163]
  We present a framework for text generation from multimodal inputs consisting of video plus text, speech, or audio. Experiments demonstrate that our approach based on a single architecture outperforms the state-of-the-art on three video-based text-generation tasks.
  arXiv  Detail & Related papers  (2021-01-28T15:22:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.