HECTOR: Hybrid Editable Compositional Object References for Video Generation

• URL: http://arxiv.org/abs/2603.08850v1
• Date: Mon, 09 Mar 2026 19:09:40 GMT
• Title: HECTOR: Hybrid Editable Compositional Object References for Video Generation
• Abstract summary: HECTOR is a generative pipeline that enables fine-grained compositional control.<n>It supports hybrid reference conditioning, allowing generation to be simultaneously guided by static images and/or dynamic videos.<n>It achieves superior visual quality, stronger reference preservation, and improved motion controllability compared with existing approaches.
• Score: 23.216951224722127
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world videos naturally portray complex interactions among distinct physical objects, effectively forming dynamic compositions of visual elements. However, most current video generation models synthesize scenes holistically and therefore lack mechanisms for explicit compositional manipulation. To address this limitation, we propose HECTOR, a generative pipeline that enables fine-grained compositional control. In contrast to prior methods,HECTOR supports hybrid reference conditioning, allowing generation to be simultaneously guided by static images and/or dynamic videos. Moreover, users can explicitly specify the trajectory of each referenced element, precisely controlling its location, scale, and speed (see Figure1). This design allows the model to synthesize coherent videos that satisfy complex spatiotemporal constraints while preserving high-fidelity adherence to references. Extensive experiments demonstrate that HECTOR achieves superior visual quality, stronger reference preservation, and improved motion controllability compared with existing approaches.

Related papers

• FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control [26.208912960696622]
  We propose a unified framework that standardizes video compositing as a trajectory-guided conditional generation task.<n>Our approach introduces three key designs: (1) a Unified Foreground Representation that decouples an object's intrinsic motion from its global displacement, standardizing heterogeneous inputs into a stabilized, centered latent space; (2) a Spatial-Aware Latent Injection strategy that exploits the translation equivariance of VAE latent spaces to transport canonical features onto target trajectories via a parameter-free mechanism; and (3) a Hybrid dataset and Synthetic-to-Real Curriculum that synergizes procedural simulation, real-world cinematic footage, and generative data to
  arXiv  Detail & Related papers  (2026-07-31T16:59:46Z)
• VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation [24.373968967561993]
  VIPER is a framework for reference-guided image-to-video generation.<n>It uses a Multimodal Large Language Model to extract reference-derived physical cues.<n>It achieves stronger reference-video physical similarity and higher human preference than representative video generation.
  arXiv  Detail & Related papers  (2026-07-26T05:55:49Z)
• Go-with-the-Track: Video Compositing and Motion Control with Point Tracking [50.22693773328114]
  Filmmaking demands precise motion control and reference image compositing.<n>We present Go-with-the-Track, which unifies both capabilities by jointly conditioning on multiple reference images and reference-anchored point-tracks.<n>Go-with-the-Track achieves superior motion and reference control in a single model.
  arXiv  Detail & Related papers  (2026-06-18T19:40:03Z)
• MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model [83.12400850233034]
  Human-Object Interaction (HOI) video reenactment with realistic motion remains a frontier in digital human creation.<n>We propose MVHOI, a two-stage HOI video reenactment framework that bridges multi-view reference conditions and video foundation models.<n>Our framework shows superior performance in generating long-duration HOI videos with intricate object manipulations.
  arXiv  Detail & Related papers  (2026-03-16T00:43:38Z)
• Compositional Video Synthesis by Temporal Object-Centric Learning [3.2228025627337864]
  We present a novel framework for compositional video synthesis that leverages temporally consistent object-centric representations.<n>Our approach explicitly captures temporal dynamics by learning pose invariant object-centric slots and conditioning them on pretrained diffusion models.<n>This design enables high-quality, pixel-level video synthesis with superior temporal coherence.
  arXiv  Detail & Related papers  (2025-07-28T14:11:04Z)
• SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios [48.09735396455107]
  Hand-Object Interaction (HOI) generation has significant application potential.<n>Current 3D HOI motion generation approaches heavily rely on predefined 3D object models and lab-captured motion data.<n>We propose a novel framework that combines visual priors and dynamic constraints within a synchronized diffusion process to generate the HOI video and motion simultaneously.
  arXiv  Detail & Related papers  (2025-06-03T05:04:29Z)
• Dimension-Reduction Attack! Video Generative Models are Experts on Controllable Image Synthesis [12.160537328404622]
  textttDRA-Ctrl provides new insights into reusing resource-intensive video models.<n>textttDRA-Ctrl lays foundation for future unified generative models across visual modalities.
  arXiv  Detail & Related papers  (2025-05-29T10:34:45Z)
• VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models [71.9811050853964]
  VideoJAM is a novel framework that instills an effective motion prior to video generators.<n>VideoJAM achieves state-of-the-art performance in motion coherence.<n>These findings emphasize that appearance and motion can be complementary and, when effectively integrated, enhance both the visual quality and the coherence of video generation.
  arXiv  Detail & Related papers  (2025-02-04T17:07:10Z)
• TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation [97.96178992465511]
  We argue that generated videos should incorporate the emergence of new concepts and their relation transitions like in real-world videos as time progresses. To assess the Temporal Compositionality of video generation models, we propose TC-Bench, a benchmark of meticulously crafted text prompts, corresponding ground truth videos, and robust evaluation metrics.
  arXiv  Detail & Related papers  (2024-06-12T21:41:32Z)
• VideoComposer: Compositional Video Synthesis with Motion Controllability [52.4714732331632]
  VideoComposer allows users to flexibly compose a video with textual conditions, spatial conditions, and more importantly temporal conditions. We introduce the motion vector from compressed videos as an explicit control signal to provide guidance regarding temporal dynamics. In addition, we develop a Spatio-Temporal Condition encoder (STC-encoder) that serves as a unified interface to effectively incorporate the spatial and temporal relations of sequential inputs.
  arXiv  Detail & Related papers  (2023-06-03T06:29:02Z)
• Multi-object Video Generation from Single Frame Layouts [84.55806837855846]
  We propose a video generative framework capable of synthesizing global scenes with local objects. Our framework is a non-trivial adaptation from image generation methods, and is new to this field. Our model has been evaluated on two widely-used video recognition benchmarks.
  arXiv  Detail & Related papers  (2023-05-06T09:07:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.