Related papers: StyleForge: Enhancing Text-to-Image Synthesis for Any Artistic Styles with Dual Binding

StyleForge: Enhancing Text-to-Image Synthesis for Any Artistic Styles with Dual Binding

URL: http://arxiv.org/abs/2404.05256v2
Date: Wed, 17 Jul 2024 06:15:10 GMT
Title: StyleForge: Enhancing Text-to-Image Synthesis for Any Artistic Styles with Dual Binding
Authors: Junseo Park, Beomseok Ko, Hyeryung Jang,
Abstract summary: We introduce Single-StyleForge, a novel approach for personalized text-to-image synthesis across diverse artistic styles. We also present Multi-StyleForge, which enhances image quality and text alignment by binding multiple tokens to partial style attributes.
Score: 7.291687946822539
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Recent advancements in text-to-image models, such as Stable Diffusion, have showcased their ability to create visual images from natural language prompts. However, existing methods like DreamBooth struggle with capturing arbitrary art styles due to the abstract and multifaceted nature of stylistic attributes. We introduce Single-StyleForge, a novel approach for personalized text-to-image synthesis across diverse artistic styles. Using approximately 15 to 20 images of the target style, Single-StyleForge establishes a foundational binding of a unique token identifier with a broad range of attributes of the target style. Additionally, auxiliary images are incorporated for dual binding that guides the consistent representation of crucial elements such as people within the target style. Furthermore, we present Multi-StyleForge, which enhances image quality and text alignment by binding multiple tokens to partial style attributes. Experimental evaluations across six distinct artistic styles demonstrate significant improvements in image quality and perceptual fidelity, as measured by FID, KID, and CLIP scores.

Related papers

Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning [11.900404048019594]
In this paper, we introduce an innovative technique to improve the quality of stylized images. Firstly, we propose Style Consistency Instance Normalization (SCIN), a method to refine the alignment between content and style features. In addition, we have developed an Instance-based Contrastive Learning (ICL) approach designed to understand relationships among various styles.
arXiv Detail & Related papers (2024-04-21T08:52:22Z)
Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization [56.12990759116612]
Pick-and-Draw is a training-free semantic guidance approach to boost identity consistency and generative diversity for personalization methods. The proposed approach can be applied to any personalized diffusion models and requires as few as a single reference image.
arXiv Detail & Related papers (2024-01-30T05:56:12Z)
PALP: Prompt Aligned Personalization of Text-to-Image Models [68.91005384187348]
Existing personalization methods compromise personalization ability or the alignment to complex prompts. We propose a new approach focusing on personalization methods for a emphsingle prompt to address this issue. Our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts.
arXiv Detail & Related papers (2024-01-11T18:35:33Z)
Style Aligned Image Generation via Shared Attention [61.121465570763085]
We introduce StyleAligned, a technique designed to establish style alignment among a series of generated images. By employing minimal attention sharing' during the diffusion process, our method maintains style consistency across images within T2I models. Our method's evaluation across diverse styles and text prompts demonstrates high-quality and fidelity.
arXiv Detail & Related papers (2023-12-04T18:55:35Z)
StyleCrafter: Enhancing Stylized Text-to-Video Generation with Style Adapter [74.68550659331405]
StyleCrafter is a generic method that enhances pre-trained T2V models with a style control adapter. To promote content-style disentanglement, we remove style descriptions from the text prompt and extract style information solely from the reference image. StyleCrafter efficiently generates high-quality stylized videos that align with the content of the texts and resemble the style of the reference images.
arXiv Detail & Related papers (2023-12-01T03:53:21Z)
DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models [11.164432246850247]
We introduce DreamStyler, a novel framework designed for artistic image synthesis. DreamStyler is proficient in both text-to-image synthesis and style transfer. With content and style guidance, DreamStyler exhibits flexibility to accommodate a range of style references.
arXiv Detail & Related papers (2023-09-13T13:13:29Z)
StyleAdapter: A Single-Pass LoRA-Free Model for Stylized Image Generation [97.24936247688824]
This paper presents a LoRA-free method for stylized image generation that takes a text prompt and style reference images as inputs. StyleAdapter can generate high-quality images that match the content of the prompts and adopt the style of the references in a single pass.
arXiv Detail & Related papers (2023-09-04T19:16:46Z)
Few-shot Font Generation by Learning Style Difference and Similarity [84.76381937516356]
We propose a novel font generation approach by learning the Difference between different styles and the Similarity of the same style (DS-Font) Specifically, we propose a multi-layer style projector for style encoding and realize a distinctive style representation via our proposed Cluster-level Contrastive Style (CCS) loss.
arXiv Detail & Related papers (2023-01-24T13:57:25Z)
Arbitrary Style Guidance for Enhanced Diffusion-Based Text-to-Image Generation [13.894251782142584]
Diffusion-based text-to-image generation models like GLIDE and DALLE-2 have gained wide success recently. We propose a novel style guidance method to support generating images using arbitrary style guided by a reference image.
arXiv Detail & Related papers (2022-11-14T20:52:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.