Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation

• URL: http://arxiv.org/abs/2606.02441v1
• Date: Mon, 01 Jun 2026 16:12:18 GMT
• Title: Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation
• Abstract summary: Identity-preserving video generation (IPVG) aims to synthesize high-fidelity videos that follow text prompts while preserving a reference identity.<n>We propose ST-DRC, an effective Spatial-Temporal Decoupled Reference Conditioning framework for identity-preserving text-to-video generation.<n>We show that ST-DRC achieves strong identity preservation, prompt alignment, temporal consistency, and video quality with a lightweight design built on LTX-2.3.
• Score: 79.94088803584262
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identity-preserving video generation (IPVG) aims to synthesize high-fidelity videos that follow text prompts while faithfully preserving a reference identity. Despite recent progress, existing IPVG methods still struggle to balance high-level semantic control and low-level identity fidelity. To bridge this gap, we propose ST-DRC, an effective Spatial-Temporal Decoupled Reference Conditioning framework for identity-preserving text-to-video generation. At the framework level, ST-DRC performs latent in-context feature injection by encoding the reference image with the video VAE and concatenating it with noisy video latents, enabling rich low-level identity details to be accessed without additional adapters. To separate identity-aware reference retrieval from appearance copying, we introduce TASS-RoPE, a Temporal-Adjacent Spatial-Shifted RoPE scheme that places reference tokens near the video sequence in time but shifts them in space, allowing reference information to flow through spatio-temporal attention while suppressing pixel-level copy-paste shortcuts. To further prevent shortcut learning and strengthen the otherwise diluted identity supervision in the diffusion objective, we combine appearance-invariant reference augmentation with face-guided identity objectives, encouraging the model to preserve identity under variations in color, pose, and layout. At inference time, we introduce a three-stream reference classifier-free guidance strategy that independently controls text adherence and reference fidelity. Experiments demonstrate that ST-DRC achieves strong identity preservation, prompt alignment, temporal consistency, and video quality with a lightweight design built on LTX-2.3. Our method ranks among the top submissions in the facial identity-preserving video generation track, validating the effectiveness of spatial-temporal decoupled reference conditioning.

Related papers

• Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generation [28.493796603648175]
  Vorch-IR is a unified framework that supports single- and dual-person identity replacement, with optional background replacement, in a single model.<n>Built on LTX2, Vorch-IR jointly conditions on a driving video, indexed reference images, and a textual editing instruction.<n>Experiments using automatic metrics and pairwise human evaluation demonstrate strong identity preservation, motion fidelity, and temporal coherence across diverse scenarios.
  arXiv  Detail & Related papers  (2026-08-06T06:46:35Z)
• UniCustom: Unified Visual Conditioning for Multi-Reference Image Generation [65.53694602893042]
  We propose a unified visual conditioning framework that fuses ViT and VAE features before VLM encoding.<n>Experiments on two multi-reference generation benchmarks demonstrate that UniCustom consistently improves subject consistency, instruction following, and compositional fidelity.
  arXiv  Detail & Related papers  (2026-05-12T13:10:05Z)
• Gloria: Consistent Character Video Generation via Content Anchors [63.7455328319917]
  We propose representing character visual attributes through a compact set of anchor frames.<n>This design provides stable references for consistency, while reference-based video generation faces challenges of copy-pasting and multi-reference conflicts.<n> Experiments show our method generates high-quality character videos exceeding 10 minutes, and achieves expressive identity and appearance consistency across views, surpassing existing methods.
  arXiv  Detail & Related papers  (2026-03-31T16:06:16Z)
• RefAlign: Representation Alignment for Reference-to-Video Generation [53.368296137314225]
  RefAlign is a representation alignment framework that aligns DiT reference-branch features to the semantic space of a visual foundation model (VFM)<n>Experiments on the OpenS2V-Eval benchmark demonstrate that RefAlign outperforms current state-of-the-art methods in TotalScore.
  arXiv  Detail & Related papers  (2026-03-26T17:59:57Z)
• AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generation [58.844504598618094]
  We propose AlcheMinT, a unified framework that introduces explicit timestamps conditioning for subject-driven video generation.<n>Our approach introduces a novel positional encoding mechanism that unlocks the encoding of temporal intervals, associated in our case with subject identities.<n>We incorporate subject-descriptive text tokens to strengthen binding between visual identity and video captions, mitigating ambiguity during generation.
  arXiv  Detail & Related papers  (2025-12-11T18:59:34Z)
• ContextAnyone: Context-Aware Diffusion for Character-Consistent Text-to-Video Generation [36.29956463871403]
  Text-to-video (T2V) generation has advanced rapidly, yet maintaining consistent character identities across scenes remains a major challenge.<n>We propose textbfContextAnyone, a context-aware diffusion framework that achieves character-consistent video generation from text and a single reference image.<n>Our method jointly reconstructs the reference image and generates new video frames, enabling the model to fully perceive and utilize reference information.
  arXiv  Detail & Related papers  (2025-12-08T09:12:18Z)
• ID-Composer: Multi-Subject Video Synthesis with Hierarchical Identity Preservation [48.59900036213667]
  Video generative models pretrained on large-scale datasets can produce high-quality videos, but are often conditioned on text or a single image.<n>We introduce ID-Composer, a novel framework that tackles multi-subject video generation from a text prompt and reference images.
  arXiv  Detail & Related papers  (2025-11-01T11:29:14Z)
• Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representations [131.33758144860988]
  Identity-preserving text-to-video (IPT2V) generation aims to create high-fidelity videos with consistent human identity.<n>Current end-to-end frameworks suffer a critical spatial-temporal trade-off.<n>We propose a simple yet effective spatial-temporal decoupled framework that decomposes representations into spatial features for layouts and temporal features for motion dynamics.
  arXiv  Detail & Related papers  (2025-07-07T06:54:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.