Beyond Facial Consistency: Personalized Person Image Generation with Holistic Identity Preservation

• URL: http://arxiv.org/abs/2607.25622v1
• Date: Tue, 28 Jul 2026 12:02:13 GMT
• Title: Beyond Facial Consistency: Personalized Person Image Generation with Holistic Identity Preservation
• Abstract summary: We propose Dynamic Balancing Scaling (DBS) to improve face and appearance identity coordination.<n>DBS consists of two components: adaptive temporal gating, and region-aware optimization.<n>We also introduce Pexels-100, a benchmark for evaluating holistic identity consistency in personalized person generation.
• Score: 31.445089759245146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Personalized person image generation requires preserving subject identity across both local facial details and broader appearance cues. Existing methods typically emphasize only one level of identity information, leading to an inherent trade-off between facial fidelity and overall appearance consistency. To address this, we first propose a simple dual-branch baseline that unifies global appearance control and local facial control within a shared generation framework. This simple combination of different branches yields promising results, but suffers from instability in practice due to uncoordinated branch contributions. To this end, we propose Dynamic Balancing Scaling (DBS), a fine-tuning strategy for improving face and appearance identity coordination. DBS consists of two components: adaptive temporal gating, which dynamically modulates branch contributions along the denoising trajectory, and region-aware optimization, which improves the coordination of facial, appearance, and global supervision. Together, these designs alleviate persistent face-branch over-dominance and encourage more effective appearance-aware guidance. We also introduce Pexels-100, a benchmark for evaluating holistic identity consistency in personalized person generation. Experiments show that DBS achieves a better trade-off between facial fidelity and appearance consistency than existing open-source baselines, while providing a controllable basic framework for holistic identity modeling.

Related papers

• Face inpainting with Identity Preserving Latent Diffusion Models [10.348748950921667]
  ID-ControlNet is an identity-preserving face inpainting framework built upon latent diffusion models.<n>We introduce an identity consistency and triplet loss training strategy that explicitly enforces alignment between the generated face and the target identity representation.
  arXiv  Detail & Related papers  (2026-05-15T23:19:43Z)
• CA-IDD: Cross-Attention Guided Identity-Conditional Diffusion for Identity-Consistent Face Swapping [1.4323566945483497]
  Face swapping aims to optimize realistic facial image generation by leveraging the identity of a source face onto a target face.<n>Existing methods, especially GAN-based methods, often struggle to balance identity preservation and visual realism.<n>We introduce CA-IDD, the first diffusion-based face swapping approach that integrates gaze, identity, and facial parsing.
  arXiv  Detail & Related papers  (2026-04-27T13:59:08Z)
• Directing the Narrative: A Finetuning Method for Controlling Coherence and Style in Story Generation [2.9255420892138306]
  We propose a two-stage framework designed for robust and consistent story generation.<n>First, we introduce Group-Shared Attention (GSA), a mechanism that fosters intrinsic consistency.<n>Second, we leverage Direct Preference Optimization (DPO) to align generated outputs with human aesthetic and narrative standards.
  arXiv  Detail & Related papers  (2026-03-18T02:43:02Z)
• Optimizing ID Consistency in Multimodal Large Models: Facial Restoration via Alignment, Entanglement, and Disentanglement [54.199726425201895]
  Multimodal editing large models have demonstrated powerful editing capabilities across diverse tasks.<n>Current facial ID preservation methods struggle to achieve consistent restoration of both facial identity and edited element IP.<n>We propose EditedID, an Alignment-Disentanglement-Entanglement framework for robust identity-specific facial restoration.
  arXiv  Detail & Related papers  (2026-02-21T08:24:42Z)
• DisCo: Reinforcement with Diversity Constraints for Multi-Human Generation [60.741022906593685]
  DisCo is the first RL-based framework to directly optimize identity diversity in multi-human generation.<n>DisCo fine-tunes flow-matching models via Group-Relative Policy Optimization.<n>On the DiverseHumans Testset, DisCo achieves 98.6 Unique Face Accuracy and near-perfect Global Identity Spread.
  arXiv  Detail & Related papers  (2025-10-01T19:28:51Z)
• PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation [62.85437763951757]
  PersonaHOI is a training- and tuning-free framework that fuses a general StableDiffusion model with a personalized face diffusion (PFD) model to generate identity-consistent human-object interaction (HOI) images.
  arXiv  Detail & Related papers  (2025-01-10T10:01:36Z)
• ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preserving [64.90148669690228]
  ConsistentID is an innovative method crafted for diverseidentity-preserving portrait generation under fine-grained multimodal facial prompts.<n>We present a fine-grained portrait dataset, FGID, with over 500,000 facial images, offering greater diversity and comprehensiveness than existing public facial datasets.
  arXiv  Detail & Related papers  (2024-04-25T17:23:43Z)
• FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping [62.38898610210771]
  We present a new single-stage method for subject face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR)
  arXiv  Detail & Related papers  (2022-10-19T11:31:38Z)
• Dual-Attention GAN for Large-Pose Face Frontalization [59.689836951934694]
  We present a novel Dual-Attention Generative Adversarial Network (DA-GAN) for photo-realistic face frontalization. Specifically, a self-attention-based generator is introduced to integrate local features with their long-range dependencies. A novel face-attention-based discriminator is applied to emphasize local features of face regions.
  arXiv  Detail & Related papers  (2020-02-17T20:00:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.