Collaborative Interactive Evolution of Art in the Latent Space of Deep Generative Models

• URL: http://arxiv.org/abs/2403.19620v1
• Date: Thu, 28 Mar 2024 17:40:15 GMT
• Title: Collaborative Interactive Evolution of Art in the Latent Space of Deep Generative Models
• Authors: Ole Hall, Anil Yaman,
• Abstract summary: We first employ GANs that are trained to produce creative images using an architecture known as Creative Adversarial Networks (CANs) We then employ an evolutionary approach to navigate within the latent space of the models to discover images. We use automatic aesthetic and collaborative interactive human evaluation metrics to assess the generated images.
• Score: 1.4425878137951238
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative Adversarial Networks (GANs) have shown great success in generating high quality images and are thus used as one of the main approaches to generate art images. However, usually the image generation process involves sampling from the latent space of the learned art representations, allowing little control over the output. In this work, we first employ GANs that are trained to produce creative images using an architecture known as Creative Adversarial Networks (CANs), then, we employ an evolutionary approach to navigate within the latent space of the models to discover images. We use automatic aesthetic and collaborative interactive human evaluation metrics to assess the generated images. In the human interactive evaluation case, we propose a collaborative evaluation based on the assessments of several participants. Furthermore, we also experiment with an intelligent mutation operator that aims to improve the quality of the images through local search based on an aesthetic measure. We evaluate the effectiveness of this approach by comparing the results produced by the automatic and collaborative interactive evolution. The results show that the proposed approach can generate highly attractive art images when the evolution is guided by collaborative human feedback.

Related papers

• KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities [93.74881034001312]
  We conduct a systematic study on the fidelity of entities in text-to-image generation models. We focus on their ability to generate a wide range of real-world visual entities, such as landmark buildings, aircraft, plants, and animals. Our findings reveal that even the most advanced text-to-image models often fail to generate entities with accurate visual details.
  arXiv  Detail & Related papers  (2024-10-15T17:50:37Z)
• Research on Optimization of Natural Language Processing Model Based on Multimodal Deep Learning [0.036651088217486416]
  This project intends to study the image representation based on attention mechanism and multimodal data. By adding multiple pattern layers to the model, the semantic and hidden layers of image content are integrated. The word vector is quantified by the Word2Vec method and then evaluated by a word embedding convolutional neural network.
  arXiv  Detail & Related papers  (2024-06-13T06:03:59Z)
• Anomaly Score: Evaluating Generative Models and Individual Generated Images based on Complexity and Vulnerability [21.355484227864466]
  We investigate the relationship between the representation space and input space around generated images. We introduce a new metric to evaluating image-generative models called anomaly score (AS)
  arXiv  Detail & Related papers  (2023-12-17T07:33:06Z)
• Improving Human-Object Interaction Detection via Virtual Image Learning [68.56682347374422]
  Human-Object Interaction (HOI) detection aims to understand the interactions between humans and objects. In this paper, we propose to alleviate the impact of such an unbalanced distribution via Virtual Image Leaning (VIL) A novel label-to-image approach, Multiple Steps Image Creation (MUSIC), is proposed to create a high-quality dataset that has a consistent distribution with real images.
  arXiv  Detail & Related papers  (2023-08-04T10:28:48Z)
• FABRIC: Personalizing Diffusion Models with Iterative Feedback [0.0]
  In an era where visual content generation is increasingly driven by machine learning, the integration of human feedback into generative models presents significant opportunities for enhancing user experience and output quality. We propose FABRIC, a training-free approach applicable to a wide range of popular diffusion models, which exploits the self-attention layer present in the most widely used architectures to condition the diffusion process on a set of feedback images. We show that generation results improve over multiple rounds of iterative feedback through exhaustive analysis, implicitly optimizing arbitrary user preferences.
  arXiv  Detail & Related papers  (2023-07-19T17:39:39Z)
• Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation [96.74302670358145]
  We introduce an automated method for Visual Concept Evaluation (ViCE) to assess consistency between a generated/edited image and the corresponding prompt/instructions. ViCE combines the strengths of Large Language Models (LLMs) and Visual Question Answering (VQA) into a unified pipeline, aiming to replicate the human cognitive process in quality assessment.
  arXiv  Detail & Related papers  (2023-07-18T16:33:30Z)
• Augmenting Character Designers Creativity Using Generative Adversarial Networks [0.0]
  Generative Adversarial Networks (GANs) continue to attract the attention of researchers in different fields. Most recent GANs are focused on realism, however, generating hyper-realistic output is not a priority for some domains. We present a comparison between different GAN architectures and their performance when trained from scratch on a new visual characters dataset. We also explore alternative techniques, such as transfer learning and data augmentation, to overcome computational resource limitations.
  arXiv  Detail & Related papers  (2023-05-28T10:52:03Z)
• Improving Synthetically Generated Image Detection in Cross-Concept Settings [20.21594285488186]
  We focus on the challenge of generalizing across different concept classes, e.g., when training a detector on human faces. We propose an approach based on the premise that the robustness of the detector can be enhanced by training it on realistic synthetic images.
  arXiv  Detail & Related papers  (2023-04-24T12:45:00Z)
• GM-NeRF: Learning Generalizable Model-based Neural Radiance Fields from Multi-view Images [79.39247661907397]
  We introduce an effective framework Generalizable Model-based Neural Radiance Fields to synthesize free-viewpoint images. Specifically, we propose a geometry-guided attention mechanism to register the appearance code from multi-view 2D images to a geometry proxy.
  arXiv  Detail & Related papers  (2023-03-24T03:32:02Z)
• Towards Creativity Characterization of Generative Models via Group-based Subset Scanning [64.6217849133164]
  We propose group-based subset scanning to identify, quantify, and characterize creative processes. We find that creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets.
  arXiv  Detail & Related papers  (2022-03-01T15:07:14Z)
• IMAGINE: Image Synthesis by Image-Guided Model Inversion [79.4691654458141]
  We introduce an inversion based method, denoted as IMAge-Guided model INvErsion (IMAGINE), to generate high-quality and diverse images. We leverage the knowledge of image semantics from a pre-trained classifier to achieve plausible generations. IMAGINE enables the synthesis procedure to simultaneously 1) enforce semantic specificity constraints during the synthesis, 2) produce realistic images without generator training, and 3) give users intuitive control over the generation process.
  arXiv  Detail & Related papers  (2021-04-13T02:00:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.