Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation

• URL: http://arxiv.org/abs/2606.01900v1
• Date: Mon, 01 Jun 2026 08:41:01 GMT
• Title: Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation
• Abstract summary: We present Auteur, a method for language-driven, human-centric camera framing in generative video.<n>Auteur enables cinematographic framing of human-centered scenes, a capability largely absent in prior generative models.
• Score: 48.49793109378558
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative video models have achieved remarkable visual fidelity and temporal coherence, yet intentional camera control remains elusive. Existing frameworks treat camera motion as a byproduct of pixel synthesis, producing trajectories that are stochastic, spatially inconsistent, and indifferent to the human subject driving the scene. In this work, we present Auteur, a method for language-driven, human-centric camera framing in generative video. Our core insight is that professional filmmakers conceive shots not as world-space trajectories but as framings defined relative to the actor, encoding shot size, angle, and composition as functions of human pose and motion. We formalize this intuition as a human-centric camera parameterization and introduce a Domain-Specific Language (DSL) that is convertible to standard 6-DoF camera parameters. A fine-tuned multimodal large language model then acts as a virtual director, mapping natural language descriptions and coarse human motion to sparse DSL keyframes that are deterministically interpolated into continuous camera trajectories, which are then provided as input to video generators. We train and evaluate Auteur on a new dataset of 34K aligned text, human motion, and DSL-annotated camera trajectories drawn from procedural synthesis and real-world movie footage from the CondensedMovies dataset. Auteur enables cinematographic framing of human-centered scenes, a capability largely absent in prior generative models. To assess this behavior, we propose new framing-focused metrics, and our experiments show that Auteur consistently outperforms existing methods

Related papers

• Towards Understanding Camera Motions in Any Video [89.97247162415158]
  We introduce CameraBench, a large-scale dataset and benchmark designed to assess and improve camera motion understanding.<n>CameraBench consists of 3,000 diverse internet videos annotated by experts through a rigorous quality control process.<n>One of our contributions is a taxonomy of camera motion primitives, designed in collaboration with cinematographers.
  arXiv  Detail & Related papers  (2025-04-21T18:34:57Z)
• GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography [98.28272367169465]
  We introduce an auto-regressive model inspired by the expertise of Directors of Photography to generate artistic and expressive camera trajectories.<n>Thanks to the comprehensive and diverse database, we train an auto-regressive, decoder-only Transformer for high-quality, context-aware camera movement generation.<n>Experiments demonstrate that compared to existing methods, GenDoP offers better controllability, finer-grained trajectory adjustments, and higher motion stability.
  arXiv  Detail & Related papers  (2025-04-09T17:56:01Z)
• ChatCam: Empowering Camera Control through Conversational AI [67.31920821192323]
  ChatCam is a system that navigates camera movements through conversations with users. To achieve this, we propose CineGPT, a GPT-based autoregressive model for text-conditioned camera trajectory generation. We also develop an Anchor Determinator to ensure precise camera trajectory placement.
  arXiv  Detail & Related papers  (2024-09-25T20:13:41Z)
• HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation [64.37874983401221]
  We present HumanVid, the first large-scale high-quality dataset tailored for human image animation. For the real-world data, we compile a vast collection of real-world videos from the internet. For the synthetic data, we collected 10K 3D avatar assets and leveraged existing assets of body shapes, skin textures and clothings.
  arXiv  Detail & Related papers  (2024-07-24T17:15:58Z)
• Decoupling Human and Camera Motion from Videos in the Wild [67.39432972193929]
  We propose a method to reconstruct global human trajectories from videos in the wild. Our method decouples the camera and human motion, which allows us to place people in the same world coordinate frame.
  arXiv  Detail & Related papers  (2023-02-24T18:59:15Z)
• HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video [44.58519508310171]
  We introduce a free-viewpoint rendering method -- HumanNeRF -- that works on a given monocular video of a human performing complex body motions. Our method enables pausing the video at any frame and rendering the subject from arbitrary new camera viewpoints.
  arXiv  Detail & Related papers  (2022-01-11T18:51:21Z)
• Batteries, camera, action! Learning a semantic control space for expressive robot cinematography [15.895161373307378]
  We develop a data-driven framework that enables editing of complex camera positioning parameters in a semantic space. First, we generate a database of video clips with a diverse range of shots in a photo-realistic simulator. We use hundreds of participants in a crowd-sourcing framework to obtain scores for a set of semantic descriptors for each clip.
  arXiv  Detail & Related papers  (2020-11-19T21:56:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.