SwiftPie: Lightning-fast Subject-driven Image Personalization via One step Diffusion

• URL: http://arxiv.org/abs/2605.01510v1
• Date: Sat, 02 May 2026 16:04:26 GMT
• Title: SwiftPie: Lightning-fast Subject-driven Image Personalization via One step Diffusion
• Abstract summary: We present SwiftPie, the first one-step diffusion image personalization tool.<n>We show that SwiftPie delivers superior image personalization speed and achieves comparable performance with multi-step approaches.
• Score: 28.837524215048187
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Diffusion models have achieved remarkable success in high-quality image synthesis, sparking interest in image-guided generation tasks such as subject-driven image personalization. Despite their impressive personalization results, existing methods typically rely on computationally intensive fine-tuning, iterative optimization, or multi-step denoising processes, which significantly hinder their deployment and interactive capability in real-time applications. In this work, we present SwiftPie, the first one-step diffusion image personalization tool that enables lightning-fast generation of personalized images. SwiftPie introduces a novel dual-branch identity injection mechanism that effectively integrates subject identity into a one-step diffusion model. In addition, we incorporate a mask-guided rescaling strategy to further enhance subject contextualization within a single diffusion step. Extensive experiments demonstrate that SwiftPie not only delivers superior image personalization speed but also achieves comparable performance with multi-step approaches in both identity fidelity and prompt alignment. This work opens new opportunities for real-time, high-quality personalized image generation, paving the way for interactive visual synthesis.

Related papers

• CopyCat: Improving Fine-Grained Subject Consistency in Subject-to-Image Models within Seconds [52.84219088497834]
  CopyCat is a lightweight model-refinement framework that improves fine-grained subject consistency within only a few seconds.<n>Experiments on DreamBench and XVerseBench demonstrate consistent improvements in fine-grained subject consistency across representative subject-to-image models.
  arXiv  Detail & Related papers  (2026-08-01T13:49:03Z)
• Zero-Shot Personalization of Objects via Textual Inversion [36.82648022347006]
  We propose a novel framework that employs a learned network to predict object-specific textual embeddings.<n>This design enables rapid, zero-shot personalization of a wide range of objects in a single forward pass.<n>To the best of our knowledge, this work represents the first attempt to achieve such general-purpose, training-free personalization within diffusion models.
  arXiv  Detail & Related papers  (2026-03-24T09:54:30Z)
• Flux Already Knows -- Activating Subject-Driven Image Generation without Training [25.496237241889048]
  We propose a zero-shot framework for subject-driven image generation using a vanilla Flux model.<n>We activate strong identity-preserving capabilities without any additional data, training, or inference-time fine-tuning.
  arXiv  Detail & Related papers  (2025-04-12T20:41:53Z)
• Fusion is all you need: Face Fusion for Customized Identity-Preserving Image Synthesis [7.099258248662009]
  Text-to-image (T2I) models have significantly advanced the development of artificial intelligence. However, existing T2I-based methods often struggle to accurately reproduce the appearance of individuals from a reference image. We leverage the pre-trained UNet from Stable Diffusion to incorporate the target face image directly into the generation process.
  arXiv  Detail & Related papers  (2024-09-27T19:31:04Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation [49.997839600988875]
  Existing personalization methods rely on finetuning a text-to-image foundation model on a user's custom dataset. We propose Joint-Image Diffusion (jedi), an effective technique for learning a finetuning-free personalization model. Our model achieves state-of-the-art generation quality, both quantitatively and qualitatively, significantly outperforming both the prior finetuning-based and finetuning-free personalization baselines.
  arXiv  Detail & Related papers  (2024-07-08T17:59:02Z)
• FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior [50.0535198082903]
  We offer a novel approach to image composition, which integrates multiple input images into a single, coherent image. We showcase the potential of utilizing the powerful generative prior inherent in large-scale pre-trained diffusion models to accomplish generic image composition.
  arXiv  Detail & Related papers  (2024-07-06T03:35:43Z)
• LCM-Lookahead for Encoder-based Text-to-Image Personalization [82.56471486184252]
  We explore the potential of using shortcut-mechanisms to guide the personalization of text-to-image models. We focus on encoder-based personalization approaches, and demonstrate that by tuning them with a lookahead identity loss, we can achieve higher identity fidelity.
  arXiv  Detail & Related papers  (2024-04-04T17:43:06Z)
• PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization [92.90392834835751]
  PortraitBooth is designed for high efficiency, robust identity preservation, and expression-editable text-to-image generation. PortraitBooth eliminates computational overhead and mitigates identity distortion. It incorporates emotion-aware cross-attention control for diverse facial expressions in generated images.
  arXiv  Detail & Related papers  (2023-12-11T13:03:29Z)
• Identity Encoder for Personalized Diffusion [57.1198884486401]
  We propose an encoder-based approach for personalization. We learn an identity encoder which can extract an identity representation from a set of reference images of a subject. We show that our approach consistently outperforms existing fine-tuning based approach in both image generation and reconstruction.
  arXiv  Detail & Related papers  (2023-04-14T23:32:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.