Opt-In Art: Learning Art Styles Only from Few Examples

• URL: http://arxiv.org/abs/2412.00176v3
• Date: Tue, 20 May 2025 19:07:27 GMT
• Title: Opt-In Art: Learning Art Styles Only from Few Examples
• Authors: Hui Ren, Joanna Materzynska, Rohit Gandikota, David Bau, Antonio Torralba,
• Abstract summary: We show that it is possible to adapt a model trained without paintings to an artistic style, given only few examples.<n>Surprisingly, our findings suggest that high-quality artistic outputs can be achieved without prior exposure to artistic data.
• Score: 50.60063523054282
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore whether pre-training on datasets with paintings is necessary for a model to learn an artistic style with only a few examples. To investigate this, we train a text-to-image model exclusively on photographs, without access to any painting-related content. We show that it is possible to adapt a model that is trained without paintings to an artistic style, given only few examples. User studies and automatic evaluations confirm that our model (post-adaptation) performs on par with state-of-the-art models trained on massive datasets that contain artistic content like paintings, drawings or illustrations. Finally, using data attribution techniques, we analyze how both artistic and non-artistic datasets contribute to generating artistic-style images. Surprisingly, our findings suggest that high-quality artistic outputs can be achieved without prior exposure to artistic data, indicating that artistic style generation can occur in a controlled, opt-in manner using only a limited, carefully selected set of training examples.

Related papers

• ArtistAuditor: Auditing Artist Style Pirate in Text-to-Image Generation Models [61.55816738318699]
  We propose a novel method for data-use auditing in the text-to-image generation model. ArtistAuditor employs a style extractor to obtain the multi-granularity style representations and treats artworks as samplings of an artist's style. The experimental results on six combinations of models and datasets show that ArtistAuditor can achieve high AUC values.
  arXiv  Detail & Related papers  (2025-04-17T16:15:38Z)
• IntroStyle: Training-Free Introspective Style Attribution using Diffusion Features [89.95303251220734]
  We present a training-free framework to solve the style attribution problem.<n>IntroStyle is shown to have superior performance to state-of-the-art models for style attribution.
  arXiv  Detail & Related papers  (2024-12-19T01:21:23Z)
• Rethinking Artistic Copyright Infringements in the Era of Text-to-Image Generative Models [47.19481598385283]
  ArtSavant is a tool to determine the unique style of an artist by comparing it to a reference dataset of works from WikiArt. We then perform a large-scale empirical study to provide quantitative insight on the prevalence of artistic style copying across 3 popular text-to-image generative models.
  arXiv  Detail & Related papers  (2024-04-11T17:59:43Z)
• AI Art Neural Constellation: Revealing the Collective and Contrastive State of AI-Generated and Human Art [36.21731898719347]
  We conduct a comprehensive analysis to position AI-generated art within the context of human art heritage. Our comparative analysis is based on an extensive dataset, dubbed ArtConstellation'' Key finding is that AI-generated artworks are visually related to the principle concepts for modern period art made in 1800-2000.
  arXiv  Detail & Related papers  (2024-02-04T11:49:51Z)
• ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank [9.99530386586636]
  Artistic style transfer aims to repaint the content image with the learned artistic style. Existing artistic style transfer methods can be divided into two categories: small model-based approaches and pre-trained large-scale model-based approaches. We propose ArtBank, a novel artistic style transfer framework, to generate highly realistic stylized images.
  arXiv  Detail & Related papers  (2023-12-11T05:53:40Z)
• Inventing art styles with no artistic training data [0.65268245109828]
  We propose two procedures to create painting styles using models trained only on natural images. In the first procedure we use the inductive bias from the artistic medium to achieve creative expression. The second procedure uses an additional natural image as inspiration to create a new style.
  arXiv  Detail & Related papers  (2023-05-19T21:59:23Z)
• Learning to Evaluate the Artness of AI-generated Images [64.48229009396186]
  ArtScore is a metric designed to evaluate the degree to which an image resembles authentic artworks by artists. We employ pre-trained models for photo and artwork generation, resulting in a series of mixed models. This dataset is then employed to train a neural network that learns to estimate quantized artness levels of arbitrary images.
  arXiv  Detail & Related papers  (2023-05-08T17:58:27Z)
• Towards mapping the contemporary art world with ArtLM: an art-specific NLP model [0.0]
  We present a generic Natural Language Processing framework (called ArtLM) to discover the connections among contemporary artists based on their biographies. With extensive experiments, we demonstrate that our ArtLM achieves 85.6% accuracy and 84.0% F1 score. We also provide a visualisation and a qualitative analysis of the artist network built from ArtLM's outputs.
  arXiv  Detail & Related papers  (2022-12-14T09:26:07Z)
• Inversion-Based Style Transfer with Diffusion Models [78.93863016223858]
  Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. We propose an inversion-based style transfer method (InST), which can efficiently and accurately learn the key information of an image.
  arXiv  Detail & Related papers  (2022-11-23T18:44:25Z)
• Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models [63.545146807810305]
  Text-to-image diffusion models can generate high-quality pictures from textual input prompts. These models have been trained using text data collected from content-based labelling protocols. We characterise the sentimentality, objectiveness and degree of abstraction of publicly available text data used to train current text-to-image diffusion models.
  arXiv  Detail & Related papers  (2022-10-19T14:20:05Z)
• Docent: A content-based recommendation system to discover contemporary art [0.8782885374383763]
  We present a content-based recommendation system on contemporary art relying on images of artworks and contextual metadata of artists. We gathered and annotated artworks with advanced and art-specific information to create a unique database that was used to train our models. After an assessment by a team of art specialists, we get an average final rating of 75% of meaningful artworks.
  arXiv  Detail & Related papers  (2022-07-12T16:26:27Z)
• Art Style Classification with Self-Trained Ensemble of AutoEncoding Transformations [5.835728107167379]
  Artistic style of a painting is a rich descriptor that reveals both visual and deep intrinsic knowledge about how an artist uniquely portrays and expresses their creative vision. In this paper, we investigate the use of deep self-supervised learning methods to solve the problem of recognizing complex artistic styles with high intra-class and low inter-class variation.
  arXiv  Detail & Related papers  (2020-12-06T21:05:23Z)
• Modeling Artistic Workflows for Image Generation and Editing [83.43047077223947]
  We propose a generative model that follows a given artistic workflow. It enables both multi-stage image generation as well as multi-stage image editing of an existing piece of art.
  arXiv  Detail & Related papers  (2020-07-14T17:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.