Rewriting Video: Text-Driven Reauthoring of Video Footage

• URL: http://arxiv.org/abs/2601.08565v1
• Date: Tue, 13 Jan 2026 13:49:05 GMT
• Title: Rewriting Video: Text-Driven Reauthoring of Video Footage
• Abstract summary: Video is a powerful medium for communication and storytelling, yet reauthoring existing footage remains challenging.<n>Recent advances in generative AI suggest a new paradigm: what if editing a video were as straightforward as rewriting text?<n>We present a tech probe and a study on text-driven video reauthoring.
• Score: 19.217919638665713
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video is a powerful medium for communication and storytelling, yet reauthoring existing footage remains challenging. Even simple edits often demand expertise, time, and careful planning, constraining how creators envision and shape their narratives. Recent advances in generative AI suggest a new paradigm: what if editing a video were as straightforward as rewriting text? To investigate this, we present a tech probe and a study on text-driven video reauthoring. Our approach involves two technical contributions: (1) a generative reconstruction algorithm that reverse-engineers video into an editable text prompt, and (2) an interactive probe, Rewrite Kit, that allows creators to manipulate these prompts. A technical evaluation of the algorithm reveals a critical human-AI perceptual gap. A probe study with 12 creators surfaced novel use cases such as virtual reshooting, synthetic continuity, and aesthetic restyling. It also highlighted key tensions around coherence, control, and creative alignment in this new paradigm. Our work contributes empirical insights into the opportunities and challenges of text-driven video reauthoring, offering design implications for future co-creative video tools.

Related papers

• A Text-Native Interface for Generative Video Authoring [5.640604615309813]
  We introduce Doki, a text-native interface for generative video authoring.<n>In Doki, writing text is the primary interaction: within a single document, users define assets, structure scenes, create shots, refine edits, and add audio.<n>This work contributes a fundamental shift in generative video interfaces, demonstrating a powerful and accessible new way to craft visual stories.
  arXiv  Detail & Related papers  (2026-03-10T01:33:41Z)
• Stealing Creator's Workflow: A Creator-Inspired Agentic Framework with Iterative Feedback Loop for Improved Scientific Short-form Generation [20.571381061542766]
  SciTalk is a novel framework for grounding videos in various sources, such as text, figures, visual styles, and avatars.<n>Inspired by content creators' iterations, SciTalk uses specialized agents for content summarization, visual scene planning, and text and layout editing.<n>Our framework provides valuable insights into the challenges and benefits of feedback-driven video generation.
  arXiv  Detail & Related papers  (2025-04-26T05:22:35Z)
• We'll Fix it in Post: Improving Text-to-Video Generation with Neuro-Symbolic Feedback [5.743225523680124]
  Current text-to-video (T2V) generation models struggle to generate semantically and temporally consistent videos when dealing with longer, more complex prompts.<n>We introduce NeuS-E, a novel zero-training video refinement pipeline that leverages neuro-symbolic feedback to automatically enhance video generation.<n>Our approach first derives the neuro-symbolic feedback by analyzing a formal video representation and pinpoints semantically inconsistent events, objects, and their corresponding frames.
  arXiv  Detail & Related papers  (2025-04-24T01:34:12Z)
• Text-Animator: Controllable Visual Text Video Generation [149.940821790235]
  We propose an innovative approach termed Text-Animator for visual text video generation. Text-Animator contains a text embedding injection module to precisely depict the structures of visual text in generated videos. We also develop a camera control module and a text refinement module to improve the stability of generated visual text.
  arXiv  Detail & Related papers  (2024-06-25T17:59:41Z)
• RACCooN: A Versatile Instructional Video Editing Framework with Auto-Generated Narratives [74.01707548681405]
  This paper proposes RACCooN, a versatile and user-friendly video-to-paragraph-to-video generative framework.<n>Our video generative model incorporates auto-generated narratives or instructions to enhance the quality and accuracy of the generated content.<n>The proposed framework demonstrates impressive versatile capabilities in video-to-paragraph generation, video content editing, and can be incorporated into other SoTA video generative models for further enhancement.
  arXiv  Detail & Related papers  (2024-05-28T17:46:36Z)
• Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools [24.71214613787985]
  The essay employs Collage as an analytical lens to analyse the user interface design of recent AI writing tools. A critical perspective relates the concerns that writers historically expressed through literary collage to AI writing tools.
  arXiv  Detail & Related papers  (2024-05-27T14:35:17Z)
• WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs [53.21307319844615]
  We present an innovative video generation AI agent that harnesses the power of Sora-inspired multimodal learning to build skilled world models framework. The framework includes two parts: prompt enhancer and full video translation.
  arXiv  Detail & Related papers  (2024-03-10T16:09:02Z)
• Neural Video Fields Editing [56.558490998753456]
  NVEdit is a text-driven video editing framework designed to mitigate memory overhead and improve consistency. We construct a neural video field, powered by tri-plane and sparse grid, to enable encoding long videos with hundreds of frames. Next, we update the video field through off-the-shelf Text-to-Image (T2I) models to text-driven editing effects.
  arXiv  Detail & Related papers  (2023-12-12T14:48:48Z)
• Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation [69.20173154096]
  We develop a framework comprised of two functional modules, Motion Structure Retrieval and Structure-Guided Text-to-Video Synthesis. For the first module, we leverage an off-the-shelf video retrieval system and extract video depths as motion structure. For the second module, we propose a controllable video generation model that offers flexible controls over structure and characters.
  arXiv  Detail & Related papers  (2023-07-13T17:57:13Z)
• Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance [36.26032505627126]
  Recent advancements in text-to-video synthesis have unveiled the potential to achieve this with prompts only. In this paper, we explore customized video generation by utilizing text as context description and motion structure. Our method, dubbed Make-Your-Video, involves joint-conditional video generation using a Latent Diffusion Model.
  arXiv  Detail & Related papers  (2023-06-01T17:43:27Z)
• Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators [70.17041424896507]
  Recent text-to-video generation approaches rely on computationally heavy training and require large-scale video datasets. We propose a new task of zero-shot text-to-video generation using existing text-to-image synthesis methods. Our method performs comparably or sometimes better than recent approaches, despite not being trained on additional video data.
  arXiv  Detail & Related papers  (2023-03-23T17:01:59Z)
• Iterative Text-based Editing of Talking-heads Using Neural Retargeting [42.964779538134714]
  We present a text-based tool for editing talking-head video that enables an iterative editing workflow. On each iteration users can edit the wording of the speech, further refine mouth motions if necessary to reduce artifacts and manipulate non-verbal aspects of the performance. Our tool requires only 2-3 minutes of the target actor video and it synthesizes the video for each iteration in about 40 seconds.
  arXiv  Detail & Related papers  (2020-11-21T01:05:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.