LensCraft: Your Professional Virtual Cinematographer

• URL: http://arxiv.org/abs/2506.00988v1
• Date: Sun, 01 Jun 2025 12:43:55 GMT
• Title: LensCraft: Your Professional Virtual Cinematographer
• Authors: Zahra Dehghanian, Morteza Abolghasemi, Hossein Azizinaghsh, Amir Vahedi, Hamid Beigy, Hamid R. Rabiee,
• Abstract summary: Digital creators, from indie filmmakers to animation studios, face a persistent bottleneck: translating their creative vision into precise camera movements.<n>LensCraft solves this problem by mimicking the expertise of a professional cinematographer, using a data-driven approach.<n>LensCraft achieves markedly lower computational complexity and faster inference while maintaining high output quality.
• Score: 12.512681517449868
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Digital creators, from indie filmmakers to animation studios, face a persistent bottleneck: translating their creative vision into precise camera movements. Despite significant progress in computer vision and artificial intelligence, current automated filming systems struggle with a fundamental trade-off between mechanical execution and creative intent. Crucially, almost all previous works simplify the subject to a single point-ignoring its orientation and true volume-severely limiting spatial awareness during filming. LensCraft solves this problem by mimicking the expertise of a professional cinematographer, using a data-driven approach that combines cinematographic principles with the flexibility to adapt to dynamic scenes in real time. Our solution combines a specialized simulation framework for generating high-fidelity training data with an advanced neural model that is faithful to the script while being aware of the volume and dynamic behavior of the subject. Additionally, our approach allows for flexible control via various input modalities, including text prompts, subject trajectory and volume, key points, or a full camera trajectory, offering creators a versatile tool to guide camera movements in line with their vision. Leveraging a lightweight real time architecture, LensCraft achieves markedly lower computational complexity and faster inference while maintaining high output quality. Extensive evaluation across static and dynamic scenarios reveals unprecedented accuracy and coherence, setting a new benchmark for intelligent camera systems compared to state-of-the-art models. Extended results, the complete dataset, simulation environment, trained model weights, and source code are publicly accessible on LensCraft Webpage.

Related papers

• ShotBench: Expert-Level Cinematic Understanding in Vision-Language Models [87.43784424444128]
  We introduce ShotBench, a benchmark specifically designed for cinematic language understanding.<n>It features over 3.5k expert-annotated QA pairs from images and video clips, meticulously curated from over 200 acclaimed (predominantly Oscar-nominated) films.<n>Our evaluation of 24 leading Vision-Language Models on ShotBench reveals their substantial limitations, particularly struggling with fine-grained visual cues and complex spatial reasoning.
  arXiv  Detail & Related papers  (2025-06-26T15:09:21Z)
• Towards Understanding Camera Motions in Any Video [80.223048294482]
  We introduce CameraBench, a large-scale dataset and benchmark designed to assess and improve camera motion understanding.<n>CameraBench consists of 3,000 diverse internet videos annotated by experts through a rigorous quality control process.<n>One of our contributions is a taxonomy of camera motion primitives, designed in collaboration with cinematographers.
  arXiv  Detail & Related papers  (2025-04-21T18:34:57Z)
• GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography [98.28272367169465]
  We introduce an auto-regressive model inspired by the expertise of Directors of Photography to generate artistic and expressive camera trajectories.<n>Thanks to the comprehensive and diverse database, we train an auto-regressive, decoder-only Transformer for high-quality, context-aware camera movement generation.<n>Experiments demonstrate that compared to existing methods, GenDoP offers better controllability, finer-grained trajectory adjustments, and higher motion stability.
  arXiv  Detail & Related papers  (2025-04-09T17:56:01Z)
• FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Video [52.33896173943054]
  Egocentric motion capture with a head-mounted body-facing stereo camera is crucial for VR and AR applications.<n>Existing methods rely on synthetic pretraining and struggle to generate smooth and accurate predictions in real-world settings.<n>We propose FRAME, a simple yet effective architecture that combines device pose and camera feeds for state-of-the-art body pose prediction.
  arXiv  Detail & Related papers  (2025-03-29T14:26:06Z)
• TexAVi: Generating Stereoscopic VR Video Clips from Text Descriptions [0.562479170374811]
  This paper proposes an approach to coalesce existing generative systems to form a stereoscopic virtual reality video from text.<n>Our work highlights the exciting possibilities of using natural language-driven graphics in fields like virtual reality simulations.
  arXiv  Detail & Related papers  (2025-01-02T09:21:03Z)
• CinePreGen: Camera Controllable Video Previsualization via Engine-powered Diffusion [29.320516135326546]
  CinePreGen is a visual previsualization system enhanced with engine-powered diffusion. It features a novel camera and storyboard interface that offers dynamic control, from global to local camera adjustments.
  arXiv  Detail & Related papers  (2024-08-30T17:16:18Z)
• Image Conductor: Precision Control for Interactive Video Synthesis [90.2353794019393]
  Filmmaking and animation production often require sophisticated techniques for coordinating camera transitions and object movements. Image Conductor is a method for precise control of camera transitions and object movements to generate video assets from a single image.
  arXiv  Detail & Related papers  (2024-06-21T17:55:05Z)
• Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis [43.02778060969546]
  We propose a controllable monocular dynamic view synthesis pipeline. Our model does not require depth as input, and does not explicitly model 3D scene geometry. We believe our framework can potentially unlock powerful applications in rich dynamic scene understanding, perception for robotics, and interactive 3D video viewing experiences for virtual reality.
  arXiv  Detail & Related papers  (2024-05-23T17:59:52Z)
• DynIBaR: Neural Dynamic Image-Based Rendering [79.44655794967741]
  We address the problem of synthesizing novel views from a monocular video depicting a complex dynamic scene. We adopt a volumetric image-based rendering framework that synthesizes new viewpoints by aggregating features from nearby views. We demonstrate significant improvements over state-of-the-art methods on dynamic scene datasets.
  arXiv  Detail & Related papers  (2022-11-20T20:57:02Z)
• Batteries, camera, action! Learning a semantic control space for expressive robot cinematography [15.895161373307378]
  We develop a data-driven framework that enables editing of complex camera positioning parameters in a semantic space. First, we generate a database of video clips with a diverse range of shots in a photo-realistic simulator. We use hundreds of participants in a crowd-sourcing framework to obtain scores for a set of semantic descriptors for each clip.
  arXiv  Detail & Related papers  (2020-11-19T21:56:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.