Customizing Video Portraits via Identity-ActionDecoupling

• URL: http://arxiv.org/abs/2606.22347v1
• Date: Sun, 21 Jun 2026 05:56:28 GMT
• Title: Customizing Video Portraits via Identity-ActionDecoupling
• Abstract summary: Identity-Preserving Text-to-Video Generation (IPT2V) seeks to synthesize a temporally coherent video from a reference image and a textual description.<n>We introduce Identity-Action Decoupling framework as well as two loss function Identity Decoupling Loss and Text Alignment Loss to solve this problem.
• Score: 32.57069491938664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identity-Preserving Text-to-Video Generation (IPT2V) seeks to synthesize a temporally coherent video from a reference image and a textual description, while simultaneously preserving the subject's identity and allowing fine-grained control over facial dynamics. Although recent methods such as ID-Animator and ConsisID inject identity features only at inference time, they ignored the ID-irrelevant information contained in Facial embedding, leading to monotonous or inaccurate facial movements that poorly follow the prompt. We introduce Identity-Action Decoupling (IaD) framework as well as two loss function Identity Decoupling Loss and Text Alignment Loss to solve this problem. Without any subject-specific fine-tuning, IaD yields videos that (1) maintain cross-temporal identity consistency and (2) exhibit rich, controllable expressions and scene variations that closely match the input text.

Related papers

• Identity-Aware Human-Object Interaction Motion Captioning [14.724174102790839]
  We introduce Identity-Aware Human-Object Interaction Motion Captioning task.<n>This task requires each generated caption to specify both the subject identity and the corresponding HOI motion.<n>We propose ID-HOINet, which learns from multi-view videos while supporting single-view identity-aware HOI motion caption generation.
  arXiv  Detail & Related papers  (2026-08-21T02:53:26Z)
• Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generation [28.493796603648175]
  Vorch-IR is a unified framework that supports single- and dual-person identity replacement, with optional background replacement, in a single model.<n>Built on LTX2, Vorch-IR jointly conditions on a driving video, indexed reference images, and a textual editing instruction.<n>Experiments using automatic metrics and pairwise human evaluation demonstrate strong identity preservation, motion fidelity, and temporal coherence across diverse scenarios.
  arXiv  Detail & Related papers  (2026-08-06T06:46:35Z)
• ID-V2V: Identity-Preserving Video Restylization [16.492389645021028]
  We formalize the challenge of identity-preserving video restylization, which propagates scene, lighting, and style changes specified by an edited video.<n>A key obstacle is the absence of paired training data, as identity-preserving restylized video pairs are rare in real-world settings.<n>We introduce ID-V2V, a video-to-video generative framework integrating complementary control signals.
  arXiv  Detail & Related papers  (2026-07-24T18:12:58Z)
• GroupVideo: Multi-Identity Customized Text-to-Video Generation [53.89036635198158]
  GroupVideo is a novel framework that leverages multiple individual photographs to generate identitycustomized video.<n>GroupVideo incorporates multimodal identity alignment, while semantic alignment introduces a semantic perceiver to enhance the naturalness of motions.<n>In response to the shortage of multi-ID video datasets, we have curated a comprehensive high-quality dataset of 20,000 videos.
  arXiv  Detail & Related papers  (2026-07-23T08:09:04Z)
• Vera: Identity-Faithful Human Subject-to-Video Generation [66.45205582225942]
  We propose Vera, a unified human-centric S2V framework for single- and multi-person generation.<n>We first construct a million-pair identity-aligned human image-video dataset through person-level cross-clip retrieval.<n>We show that Vera improves human identity consistency, multi-person subject binding, and motion naturalness, while reducing identity confusion and excessive reference-image copying.
  arXiv  Detail & Related papers  (2026-07-22T15:08:30Z)
• AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation [55.94507360511886]
  We propose AnyCrowd, a Transformer-based video generation framework capable of scaling to an arbitrary number of characters.<n>Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement.<n>Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global-background coordination.
  arXiv  Detail & Related papers  (2026-03-16T15:25:04Z)
• Optimizing ID Consistency in Multimodal Large Models: Facial Restoration via Alignment, Entanglement, and Disentanglement [54.199726425201895]
  Multimodal editing large models have demonstrated powerful editing capabilities across diverse tasks.<n>Current facial ID preservation methods struggle to achieve consistent restoration of both facial identity and edited element IP.<n>We propose EditedID, an Alignment-Disentanglement-Entanglement framework for robust identity-specific facial restoration.
  arXiv  Detail & Related papers  (2026-02-21T08:24:42Z)
• AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generation [58.844504598618094]
  We propose AlcheMinT, a unified framework that introduces explicit timestamps conditioning for subject-driven video generation.<n>Our approach introduces a novel positional encoding mechanism that unlocks the encoding of temporal intervals, associated in our case with subject identities.<n>We incorporate subject-descriptive text tokens to strengthen binding between visual identity and video captions, mitigating ambiguity during generation.
  arXiv  Detail & Related papers  (2025-12-11T18:59:34Z)
• BeyondFacial: Identity-Preserving Personalized Generation Beyond Facial Close-ups [22.017690133402912]
  Identity-Preserving Personalized Generation (I) has advanced film production and artistic creation, yet existing approaches overemphasize facial regions.<n>These methods suffer from weak visual narrativity and poor semantic consistency under complex text prompts.<n>This paper presents an I method that breaks the constraint of facial close-ups, achieving synergistic optimization of identity fidelity and scene semantic creation.
  arXiv  Detail & Related papers  (2025-11-15T01:56:14Z)
• Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization [55.29071072675132]
  Face anonymization aims to conceal identity information while preserving non-identity attributes.<n>We propose textbfIDsuperscript2Face, a training-centric anonymization framework.<n>We show that IDtextsuperscript2Face outperforms existing methods in visual quality, identity suppression, and utility preservation.
  arXiv  Detail & Related papers  (2025-10-28T09:28:12Z)
• Mask-Free Audio-driven Talking Face Generation for Enhanced Visual Quality and Identity Preservation [54.52905471078152]
  We propose a mask-free talking face generation approach while maintaining the 2D-based face editing task.<n>We transform the input images to have closed mouths, using a two-step landmark-based approach trained in an unpaired manner.
  arXiv  Detail & Related papers  (2025-07-28T16:03:36Z)
• ID-EA: Identity-driven Text Enhancement and Adaptation with Textual Inversion for Personalized Text-to-Image Generation [33.84646269805187]
  ID-EA is a novel framework that guides text embeddings to align with visual identity embeddings.<n> ID-EA substantially outperforms state-of-the-art methods in identity preservation metrics.<n>It generates personalized portraits 15 times faster than existing approaches.
  arXiv  Detail & Related papers  (2025-07-16T07:42:02Z)
• Proteus-ID: ID-Consistent and Motion-Coherent Video Customization [17.792780924370103]
  Video identity customization seeks to synthesize realistic, temporally coherent videos of a specific subject, given a single reference image and a text prompt.<n>This task presents two core challenges: maintaining identity consistency while aligning with the described appearance and actions, and generating natural, fluid motion without unrealistic stiffness.<n>We introduce Proteus-ID, a novel diffusion-based framework for identity-consistent and motion-coherent video customization.
  arXiv  Detail & Related papers  (2025-06-30T11:05:32Z)
• EmojiDiff: Advanced Facial Expression Control with High Identity Preservation in Portrait Generation [8.314556078632412]
  We introduce EmojiDiff, the first end-to-end solution that enables simultaneous control of extremely detailed expression (RGB-level) and high-fidelity identity in portrait generation.<n>For decoupled training, we innovate ID-irrelevant Data Iteration (IDI) to synthesize cross-identity expression pairs.<n>We also present ID-enhanced Contrast Alignment (ICA) for further fine-tuning.
  arXiv  Detail & Related papers  (2024-12-02T08:24:11Z)
• PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation [36.21554597804604]
  Identity-specific human video generation with customized ID images is still under-explored.<n>Key challenge lies in maintaining high ID fidelity consistently while preserving the original motion dynamic and semantic following.<n>We propose a novel framework, dubbed $textbfPersonalVideo$, that applies a mixture of reward supervision on synthesized videos.
  arXiv  Detail & Related papers  (2024-11-26T02:25:38Z)
• Identity-Aware Multi-Sentence Video Description [105.13845996039277]
  We introduce an auxiliary task of Fill-in the Identity, that aims to predict persons' IDs consistently within a set of clips. One of the key components is a gender-aware textual representation as well an additional gender prediction objective in the main model. Experiments show that our proposed Fill-in the Identity model is superior to several baselines and recent works.
  arXiv  Detail & Related papers  (2020-08-22T09:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.