Personalized Image Generation from an Author Writing Style

• URL: http://arxiv.org/abs/2507.03313v1
• Date: Fri, 04 Jul 2025 05:53:48 GMT
• Title: Personalized Image Generation from an Author Writing Style
• Authors: Sagar Gandhi, Vishal Gandhi,
• Abstract summary: Translating nuanced, textually-defined authorial writing styles into compelling visual representations presents a novel challenge in generative AI.<n>This paper introduces a pipeline that leverages Author Writing Sheets (AWS) as input to a Large Language Model (LLM)<n>We evaluated our approach using 49 author styles from Reddit data, with human evaluators assessing the stylistic match and visual distinctiveness of the generated images.
• Score: 0.29998889086656577
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Translating nuanced, textually-defined authorial writing styles into compelling visual representations presents a novel challenge in generative AI. This paper introduces a pipeline that leverages Author Writing Sheets (AWS) - structured summaries of an author's literary characteristics - as input to a Large Language Model (LLM, Claude 3.7 Sonnet). The LLM interprets the AWS to generate three distinct, descriptive text-to-image prompts, which are then rendered by a diffusion model (Stable Diffusion 3.5 Medium). We evaluated our approach using 49 author styles from Reddit data, with human evaluators assessing the stylistic match and visual distinctiveness of the generated images. Results indicate a good perceived alignment between the generated visuals and the textual authorial profiles (mean style match: $4.08/5$), with images rated as moderately distinctive. Qualitative analysis further highlighted the pipeline's ability to capture mood and atmosphere, while also identifying challenges in representing highly abstract narrative elements. This work contributes a novel end-to-end methodology for visual authorial style personalization and provides an initial empirical validation, opening avenues for applications in creative assistance and cross-modal understanding.

Related papers

• DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models [115.62816053600085]
  We present DesignDiffusion, a framework for synthesizing design images from textual descriptions.<n>The proposed framework directly synthesizes textual and visual design elements from user prompts.<n>It utilizes a distinctive character embedding derived from the visual text to enhance the input prompt.
  arXiv  Detail & Related papers  (2025-03-03T15:22:57Z)
• Towards Visual Text Design Transfer Across Languages [49.78504488452978]
  We introduce a novel task of Multimodal Style Translation (MuST-Bench) MuST-Bench is a benchmark designed to evaluate the ability of visual text generation models to perform translation across different writing systems. In response, we introduce SIGIL, a framework for multimodal style translation that eliminates the need for style descriptions.
  arXiv  Detail & Related papers  (2024-10-24T15:15:01Z)
• VitaGlyph: Vitalizing Artistic Typography with Flexible Dual-branch Diffusion Models [53.59400446543756]
  Artistic typography is a technique to visualize the meaning of input character in an imaginable and readable manner.<n>We introduce a dual-branch, training-free method called VitaGlyph, enabling flexible artistic typography with controllable geometry changes.
  arXiv  Detail & Related papers  (2024-10-02T16:48:47Z)
• Capturing Style in Author and Document Representation [4.323709559692927]
  We propose a new architecture that learns embeddings for both authors and documents with a stylistic constraint.<n>We evaluate our method on three datasets: a literary corpus extracted from the Gutenberg Project, the Blog Authorship and IMDb62.
  arXiv  Detail & Related papers  (2024-07-18T10:01:09Z)
• Text-guided Image Restoration and Semantic Enhancement for Text-to-Image Person Retrieval [12.057465578064345]
  The goal of Text-to-Image Person Retrieval (TIPR) is to retrieve specific person images according to the given textual descriptions.<n>We propose a novel TIPR framework to build fine-grained interactions and alignment between person images and the corresponding texts.
  arXiv  Detail & Related papers  (2023-07-18T08:23:46Z)
• ALADIN-NST: Self-supervised disentangled representation learning of artistic style through Neural Style Transfer [60.6863849241972]
  We learn a representation of visual artistic style more strongly disentangled from the semantic content depicted in an image. We show that strongly addressing the disentanglement of style and content leads to large gains in style-specific metrics.
  arXiv  Detail & Related papers  (2023-04-12T10:33:18Z)
• WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models [8.334487584550185]
  We present a latent diffusion-based method for styled text-to-text-content-image generation on word-level. Our proposed method is able to generate realistic word image samples from different writer styles. We show that the proposed model produces samples that are aesthetically pleasing, help boosting text recognition performance, and get similar writer retrieval score as real data.
  arXiv  Detail & Related papers  (2023-03-29T10:19:26Z)
• DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion [10.75789076591325]
  We introduce a novel method to automatically generate an artistic typography by stylizing one or more letter fonts. Our approach utilizes large language models to bridge texts and visual images for stylization and build an unsupervised generative model.
  arXiv  Detail & Related papers  (2023-03-16T19:12:52Z)
• Style-Aware Contrastive Learning for Multi-Style Image Captioning [36.1319565907582]
  We present a style-aware visual encoder with contrastive learning to mine potential visual content relevant to style. We also propose a style-aware triplet contrast objective to distinguish whether the image, style and caption matched. Experimental results demonstrate that our approach achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-01-26T19:21:39Z)
• Holistic Visual-Textual Sentiment Analysis with Prior Models [64.48229009396186]
  We propose a holistic method that achieves robust visual-textual sentiment analysis. The proposed method consists of four parts: (1) a visual-textual branch to learn features directly from data for sentiment analysis, (2) a visual expert branch with a set of pre-trained "expert" encoders to extract selected semantic visual features, (3) a CLIP branch to implicitly model visual-textual correspondence, and (4) a multimodal feature fusion network based on BERT to fuse multimodal features and make sentiment predictions.
  arXiv  Detail & Related papers  (2022-11-23T14:40:51Z)
• Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets [56.018551958004814]
  This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources. Large-scale datasets with noisy image-text pairs provide a sub-optimal source of supervision. We propose to leverage and separate semantics and descriptive style through the incorporation of a style token and keywords extracted through a retrieval component.
  arXiv  Detail & Related papers  (2021-11-24T19:00:05Z)
• Improving Generation and Evaluation of Visual Stories via Semantic Consistency [72.00815192668193]
  Given a series of natural language captions, an agent must generate a sequence of images that correspond to the captions. Prior work has introduced recurrent generative models which outperform synthesis text-to-image models on this task. We present a number of improvements to prior modeling approaches, including the addition of a dual learning framework.
  arXiv  Detail & Related papers  (2021-05-20T20:42:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.