The role of interface design on prompt-mediated creativity in Generative AI

• URL: http://arxiv.org/abs/2312.00233v2
• Date: Sat, 17 Feb 2024 13:29:10 GMT
• Title: The role of interface design on prompt-mediated creativity in Generative AI
• Authors: Maddalena Torricelli, Mauro Martino, Andrea Baronchelli, Luca Maria Aiello
• Abstract summary: We analyze more than 145,000 prompts from two Generative AI platforms. We find that users exhibit a tendency towards exploration of new topics over exploitation of concepts visited previously.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative AI for the creation of images is becoming a staple in the toolkit of digital artists and visual designers. The interaction with these systems is mediated by \emph{prompting}, a process in which users write a short text to describe the desired image's content and style. The study of prompts offers an unprecedented opportunity to gain insight into the process of human creativity. Yet, our understanding of how people use them remains limited. We analyze more than 145,000 prompts from the logs of two Generative AI platforms (Stable Diffusion and Pick-a-Pic) to shed light on how people \emph{explore} new concepts over time, and how their exploration might be influenced by different design choices in human-computer interfaces to Generative AI. We find that users exhibit a tendency towards exploration of new topics over exploitation of concepts visited previously. However, a comparative analysis of the two platforms, which differ both in scope and functionalities, reveals some stark differences. Features diverting user focus from prompting and providing instead shortcuts for quickly generating image variants are associated with a considerable reduction in both exploration of novel concepts and detail in the submitted prompts. These results carry direct implications for the design of human interfaces to Generative AI and raise new questions regarding how the process of prompting should be aided in ways that best support creativity.

Related papers

• Survey of User Interface Design and Interaction Techniques in Generative AI Applications [79.55963742878684]
  We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-10-28T23:10:06Z)
• Diffusion-Based Visual Art Creation: A Survey and New Perspectives [51.522935314070416]
  This survey explores the emerging realm of diffusion-based visual art creation, examining its development from both artistic and technical perspectives. Our findings reveal how artistic requirements are transformed into technical challenges and highlight the design and application of diffusion-based methods within visual art creation. We aim to shed light on the mechanisms through which AI systems emulate and possibly, enhance human capacities in artistic perception and creativity.
  arXiv  Detail & Related papers  (2024-08-22T04:49:50Z)
• Inspired by AI? A Novel Generative AI System To Assist Conceptual Automotive Design [6.001793288867721]
  Design inspiration is crucial for establishing the direction of a design as well as evoking feelings and conveying meanings during the conceptual design process. Many practice designers use text-based searches on platforms like Pinterest to gather image ideas, followed by sketching on paper or using digital tools to develop concepts. Emerging generative AI techniques, such as diffusion models, offer a promising avenue to streamline these processes by swiftly generating design concepts based on text and image inspiration inputs.
  arXiv  Detail & Related papers  (2024-06-06T17:04:14Z)
• VISHIEN-MAAT: Scrollytelling visualization design for explaining Siamese Neural Network concept to non-technical users [8.939421900877742]
  This work proposes a novel visualization design for creating a scrollytelling that can effectively explain an AI concept to non-technical users. Our approach helps create a visualization valuable for a short-timeline situation like a sales pitch.
  arXiv  Detail & Related papers  (2023-04-04T08:26:54Z)
• How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models [29.420160518026496]
  We discuss the opportunities and challenges for interactive creative applications that use prompting as a new paradigm for Human-AI interaction. Based on our analysis, we propose four design goals for user interfaces that support prompting. We illustrate these with concrete UI design sketches, focusing on the use case of creative writing.
  arXiv  Detail & Related papers  (2022-09-03T10:16:34Z)
• Pathway to Future Symbiotic Creativity [76.20798455931603]
  We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist to a Machine artist in its own right. In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations. We propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle.
  arXiv  Detail & Related papers  (2022-08-18T15:12:02Z)
• A Taxonomy of Prompt Modifiers for Text-To-Image Generation [6.903929927172919]
  This paper identifies six types of prompt modifier used by practitioners in the online community based on a 3-month ethnography study. The novel taxonomy of prompt modifier provides researchers a conceptual starting point for investigating the practice of text-to-image generation. We discuss research opportunities of this novel creative practice in the field of Human-Computer Interaction.
  arXiv  Detail & Related papers  (2022-04-20T06:15:50Z)
• SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning [61.57887011165744]
  multimodal Transformers have made great progress in the task of Visual Commonsense Reasoning. We propose a Scene Graph Enhanced Image-Text Learning framework to incorporate visual scene graphs in commonsense reasoning.
  arXiv  Detail & Related papers  (2021-12-16T03:16:30Z)
• Telling Creative Stories Using Generative Visual Aids [52.623545341588304]
  We asked writers to write creative stories from a starting prompt, and provided them with visuals created by generative AI models from the same prompt. Compared to a control group, writers who used the visuals as story writing aid wrote significantly more creative, original, complete and visualizable stories. Findings indicate that cross modality inputs by AI can benefit divergent aspects of creativity in human-AI co-creation, but hinders convergent thinking.
  arXiv  Detail & Related papers  (2021-10-27T23:13:47Z)
• Human in the Loop for Machine Creativity [0.0]
  We conceptualize existing and future human-in-the-loop (HITL) approaches for creative applications. We examine and speculate on long term implications for models, interfaces, and machine creativity. We envision multimodal HITL processes, where texts, visuals, sounds, and other information are coupled together, with automated analysis of humans and environments.
  arXiv  Detail & Related papers  (2021-10-07T15:42:18Z)
• This is not the Texture you are looking for! Introducing Novel Counterfactual Explanations for Non-Experts using Generative Adversarial Learning [59.17685450892182]
  counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image. We present a novel approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques. Our results show that our approach leads to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the art systems.
  arXiv  Detail & Related papers  (2020-12-22T10:08:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.