CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Models

• URL: http://arxiv.org/abs/2508.20640v1
• Date: Thu, 28 Aug 2025 10:38:13 GMT
• Title: CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Models
• Authors: Ayan Banerjee, Fernando Vilariño, Josep Lladós,
• Abstract summary: Preserving facial identity under extreme stylistic transformation remains a major challenge in generative art.<n>We present CraftGraffiti, an end-to-end text-guided graffiti generation framework designed with facial feature preservation as a primary objective.<n>CraftGraffiti first applies graffiti style transfer via LoRA-fine-tuned pretrained diffusion transformer, then enforces identity fidelity through a face-consistent self-attention mechanism.
• Score: 43.96017763034248
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Preserving facial identity under extreme stylistic transformation remains a major challenge in generative art. In graffiti, a high-contrast, abstract medium, subtle distortions to the eyes, nose, or mouth can erase the subject's recognizability, undermining both personal and cultural authenticity. We present CraftGraffiti, an end-to-end text-guided graffiti generation framework designed with facial feature preservation as a primary objective. Given an input image and a style and pose descriptive prompt, CraftGraffiti first applies graffiti style transfer via LoRA-fine-tuned pretrained diffusion transformer, then enforces identity fidelity through a face-consistent self-attention mechanism that augments attention layers with explicit identity embeddings. Pose customization is achieved without keypoints, using CLIP-guided prompt extension to enable dynamic re-posing while retaining facial coherence. We formally justify and empirically validate the "style-first, identity-after" paradigm, showing it reduces attribute drift compared to the reverse order. Quantitative results demonstrate competitive facial feature consistency and state-of-the-art aesthetic and human preference scores, while qualitative analyses and a live deployment at the Cruilla Festival highlight the system's real-world creative impact. CraftGraffiti advances the goal of identity-respectful AI-assisted artistry, offering a principled approach for blending stylistic freedom with recognizability in creative AI applications.

Related papers

• SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy [53.75084833636302]
  We propose SIDeR, a Semantic decoupling-driven framework for unrestricted face privacy protection.<n> SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component.<n>For authorized access, SIDeR can be restored to its original form when the correct password is provided.
  arXiv  Detail & Related papers  (2026-02-04T19:30:48Z)
• Advancing Facial Stylization through Semantic Preservation Constraint and Pseudo-Paired Supervision [3.4228848885035092]
  We argue that these issues stem from neglecting semantic shift of the generator during stylization.<n>We propose a facial stylization method that integrates semantic preservation constraint and pseudo-paired supervision.<n>Building upon our facial stylization framework, we achieve more flexible multimodal and reference-guided stylization.
  arXiv  Detail & Related papers  (2025-06-27T08:44:31Z)
• Training-Free Diffusion Framework for Stylized Image Generation with Identity Preservation [0.31498833540989407]
  "Mosaic Restored Content Image" technique significantly enhances identity retention in complex scenes.<n>"Mosaic Restored Content Image" technique improves the preservation of fine-grained details by directing more attention to the original image during stylization.
  arXiv  Detail & Related papers  (2025-06-07T13:54:02Z)
• InstaFace: Identity-Preserving Facial Editing with Single Image Inference [13.067402877443902]
  We introduce a novel diffusion-based framework, InstaFace, to generate realistic images while preserving identity using only a single image.<n>InstaFace harnesses 3D perspectives by integrating multiple 3DMM-based conditionals without introducing additional trainable parameters.<n>Our method outperforms several state-of-the-art approaches in terms of identity preservation, photorealism, and effective control of pose, expression, and lighting.
  arXiv  Detail & Related papers  (2025-02-27T22:37:09Z)
• Foundation Cures Personalization: Improving Personalized Models' Prompt Consistency via Hidden Foundation Knowledge [33.35678923549471]
  textbfFreeCure is a framework that improves the prompt consistency of personalization models.<n>We introduce a novel foundation-aware self-attention module, coupled with an inversion-based process to bring well-aligned attribute information to the personalization process.<n>FreeCure has demonstrated significant improvements in prompt consistency across a diverse set of state-of-the-art facial personalization models.
  arXiv  Detail & Related papers  (2024-11-22T15:21:38Z)
• Content and Salient Semantics Collaboration for Cloth-Changing Person Re-Identification [74.10897798660314]
  Cloth-changing person re-identification aims at recognizing the same person with clothing changes across non-overlapping cameras.<n>We propose a unified Semantics Mining and Refinement (SMR) module to extract robust identity-related content and salient semantics, mitigating interference from clothing appearances effectively.<n>Our proposed method achieves state-of-the-art performance on three cloth-changing benchmarks, demonstrating its superiority over advanced competitors.
  arXiv  Detail & Related papers  (2024-05-26T15:17:28Z)
• DiffFAE: Advancing High-fidelity One-shot Facial Appearance Editing with Space-sensitive Customization and Semantic Preservation [84.0586749616249]
  This paper presents DiffFAE, a one-stage and highly-efficient diffusion-based framework tailored for high-fidelity Facial Appearance Editing. For high-fidelity query attributes transfer, we adopt Space-sensitive Physical Customization (SPC), which ensures the fidelity and generalization ability. In order to preserve source attributes, we introduce the Region-responsive Semantic Composition (RSC) This module is guided to learn decoupled source-regarding features, thereby better preserving the identity and alleviating artifacts from non-facial attributes such as hair, clothes, and background.
  arXiv  Detail & Related papers  (2024-03-26T12:53:10Z)
• DemoCaricature: Democratising Caricature Generation with a Rough Sketch [80.90808879991182]
  We democratise caricature generation, empowering individuals to craft personalised caricatures with just a photo and a conceptual sketch. Our objective is to strike a delicate balance between abstraction and identity, while preserving the creativity and subjectivity inherent in a sketch.
  arXiv  Detail & Related papers  (2023-12-07T15:35:42Z)
• Personalized Face Inpainting with Diffusion Models by Parallel Visual Attention [55.33017432880408]
  This paper proposes the use of Parallel Visual Attention (PVA) in conjunction with diffusion models to improve inpainting results. We train the added attention modules and identity encoder on CelebAHQ-IDI, a dataset proposed for identity-preserving face inpainting. Experiments demonstrate that PVA attains unparalleled identity resemblance in both face inpainting and face inpainting with language guidance tasks.
  arXiv  Detail & Related papers  (2023-12-06T15:39:03Z)
• When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for Personalized Image Generation [60.305112612629465]
  Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images. We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models. Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
  arXiv  Detail & Related papers  (2023-11-29T09:05:14Z)
• Few-shots Portrait Generation with Style Enhancement and Identity Preservation [3.6937810031393123]
  StyleIdentityGAN model can ensure the identity and artistry of the generated portrait at the same time. Style-enhanced module focuses on artistic style features decoupling and transferring to improve the artistry of generated virtual face images. Experiments demonstrate the superiority of StyleIdentityGAN over state-of-art methods in artistry and identity effects.
  arXiv  Detail & Related papers  (2023-03-01T10:02:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.