ID-EA: Identity-driven Text Enhancement and Adaptation with Textual Inversion for Personalized Text-to-Image Generation

• URL: http://arxiv.org/abs/2507.11990v1
• Date: Wed, 16 Jul 2025 07:42:02 GMT
• Title: ID-EA: Identity-driven Text Enhancement and Adaptation with Textual Inversion for Personalized Text-to-Image Generation
• Authors: Hyun-Jun Jin, Young-Eun Kim, Seong-Whan Lee,
• Abstract summary: ID-EA is a novel framework that guides text embeddings to align with visual identity embeddings.<n> ID-EA substantially outperforms state-of-the-art methods in identity preservation metrics.<n>It generates personalized portraits 15 times faster than existing approaches.
• Score: 33.84646269805187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, personalized portrait generation with a text-to-image diffusion model has significantly advanced with Textual Inversion, emerging as a promising approach for creating high-fidelity personalized images. Despite its potential, current Textual Inversion methods struggle to maintain consistent facial identity due to semantic misalignments between textual and visual embedding spaces regarding identity. We introduce ID-EA, a novel framework that guides text embeddings to align with visual identity embeddings, thereby improving identity preservation in a personalized generation. ID-EA comprises two key components: the ID-driven Enhancer (ID-Enhancer) and the ID-conditioned Adapter (ID-Adapter). First, the ID-Enhancer integrates identity embeddings with a textual ID anchor, refining visual identity embeddings derived from a face recognition model using representative text embeddings. Then, the ID-Adapter leverages the identity-enhanced embedding to adapt the text condition, ensuring identity preservation by adjusting the cross-attention module in the pre-trained UNet model. This process encourages the text features to find the most related visual clues across the foreground snippets. Extensive quantitative and qualitative evaluations demonstrate that ID-EA substantially outperforms state-of-the-art methods in identity preservation metrics while achieving remarkable computational efficiency, generating personalized portraits approximately 15 times faster than existing approaches.

Related papers

• EditID: Training-Free Editable ID Customization for Text-to-Image Generation [12.168520751389622]
  We propose EditID, a training-free approach based on the DiT architecture, which achieves highly editable customized IDs for text to image generation.<n>It is challenging to alter facial orientation, character attributes, and other features through prompts.<n> EditID is the first text-to-image solution to propose customizable ID editability on the DiT architecture.
  arXiv  Detail & Related papers  (2025-03-16T14:41:30Z)
• DynamicID: Zero-Shot Multi-ID Image Personalization with Flexible Facial Editability [12.692129257068085]
  We present DynamicID, a tuning-free framework that inherently facilitates both single-ID and multi-ID personalized generation.<n>Our key innovations include: 1) Semantic-Activated Attention (SAA), which employs query-level activation gating to minimize disruption to the base model when injecting ID features and achieve multi-ID personalization without requiring multi-ID samples during training; 2) Identity-Motion Reconfigurator (IMR), which applies feature-space manipulation to effectively disentangle facial motion and identity features, supporting flexible facial editing; and 3) a task-decoupled training paradigm that reduces data dependency, together with VariFace-10k,
  arXiv  Detail & Related papers  (2025-03-09T08:16:19Z)
• ChatReID: Open-ended Interactive Person Retrieval via Hierarchical Progressive Tuning for Vision Language Models [49.09606704563898]
  Person re-identification is a crucial task in computer vision, aiming to recognize individuals across non-overlapping camera views.<n>We propose a novel framework ChatReID, that shifts the focus towards a text-side-dominated retrieval paradigm, enabling flexible and interactive re-identification.<n>We introduce a hierarchical progressive tuning strategy, which endows Re-ID ability through three stages of tuning, i.e., from person attribute understanding to fine-grained image retrieval and to multi-modal task reasoning.
  arXiv  Detail & Related papers  (2025-02-27T10:34:14Z)
• See What You Seek: Semantic Contextual Integration for Cloth-Changing Person Re-Identification [14.01260112340177]
  Cloth-changing person re-identification (CC-ReID) aims to match individuals across surveillance cameras despite variations in clothing.<n>Existing methods typically mitigate the impact of clothing changes or enhance identity (ID)-relevant features.<n>We propose a novel prompt learning framework Semantic Contextual Integration (SCI) to reduce clothing-induced discrepancies and strengthen ID cues.
  arXiv  Detail & Related papers  (2024-12-02T10:11:16Z)
• CustAny: Customizing Anything from A Single Example [73.90939022698399]
  We present a novel pipeline to construct a large dataset of general objects, featuring 315k text-image samples across 10k categories. With the help of MC-IDC, we introduce Customizing Anything (CustAny), a zero-shot framework that maintains ID fidelity and supports flexible text editing for general objects. Our contributions include a large-scale dataset, the CustAny framework and novel ID processing to advance this field.
  arXiv  Detail & Related papers  (2024-06-17T15:26:22Z)
• ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning [57.91881829308395]
  Identity-preserving text-to-image generation (ID-T2I) has received significant attention due to its wide range of application scenarios like AI portrait and advertising. We present textbfID-Aligner, a general feedback learning framework to enhance ID-T2I performance.
  arXiv  Detail & Related papers  (2024-04-23T18:41:56Z)
• Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm [31.06269858216316]
  We propose Infinite-ID, an ID-semantics decoupling paradigm for identity-preserved personalization. We introduce an identity-enhanced training, incorporating an additional image cross-attention module to capture sufficient ID information. We also introduce a feature interaction mechanism that combines a mixed attention module with an AdaIN-mean operation to seamlessly merge the two streams.
  arXiv  Detail & Related papers  (2024-03-18T13:39:53Z)
• Beyond Inserting: Learning Identity Embedding for Semantic-Fidelity Personalized Diffusion Generation [21.739328335601716]
  This paper focuses on inserting accurate and interactive ID embedding into the Stable Diffusion Model for personalized generation. We propose a face-wise attention loss to fit the face region instead of entangling ID-unrelated information, such as face layout and background. Our results exhibit superior ID accuracy, text-based manipulation ability, and generalization compared to previous methods.
  arXiv  Detail & Related papers  (2024-01-31T11:52:33Z)
• StableIdentity: Inserting Anybody into Anywhere at First Sight [57.99693188913382]
  We propose StableIdentity, which allows identity-consistent recontextualization with just one face image. We are the first to directly inject the identity learned from a single image into video/3D generation without finetuning.
  arXiv  Detail & Related papers  (2024-01-29T09:06:15Z)
• PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding [102.07914175196817]
  PhotoMaker is an efficient personalized text-to-image generation method. It encodes an arbitrary number of input ID images into a stack ID embedding for preserving ID information.
  arXiv  Detail & Related papers  (2023-12-07T17:32:29Z)
• When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for Personalized Image Generation [60.305112612629465]
  Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images. We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models. Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
  arXiv  Detail & Related papers  (2023-11-29T09:05:14Z)
• FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping [62.38898610210771]
  We present a new single-stage method for subject face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR)
  arXiv  Detail & Related papers  (2022-10-19T11:31:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.