Exploring the Potential of Large Language Models in Artistic Creation:
Collaboration and Reflection on Creative Programming
- URL: http://arxiv.org/abs/2402.09750v1
- Date: Thu, 15 Feb 2024 07:00:06 GMT
- Title: Exploring the Potential of Large Language Models in Artistic Creation:
Collaboration and Reflection on Creative Programming
- Authors: Anqi Wang, Zhizhuo Yin, Yulu Hu, Yuanyuan Mao, Pan Hui
- Abstract summary: We compare two common collaboration approaches: invoking the entire program and multiple subtasks.
Our findings exhibit artists' different stimulated reflections in two different methods.
Our work reveals the artistic potential of LLM in creative coding.
- Score: 10.57792673254363
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Recently, the potential of large language models (LLMs) has been widely used
in assisting programming. However, current research does not explore the artist
potential of LLMs in creative coding within artist and AI collaboration. Our
work probes the reflection type of artists in the creation process with such
collaboration. We compare two common collaboration approaches: invoking the
entire program and multiple subtasks. Our findings exhibit artists' different
stimulated reflections in two different methods. Our finding also shows the
correlation of reflection type with user performance, user satisfaction, and
subjective experience in two collaborations through conducting two methods,
including experimental data and qualitative interviews. In this sense, our work
reveals the artistic potential of LLM in creative coding. Meanwhile, we provide
a critical lens of human-AI collaboration from the artists' perspective and
expound design suggestions for future work of AI-assisted creative tasks.
Related papers
- Equivalence: An analysis of artists' roles with Image Generative AI from Conceptual Art perspective through an interactive installation design practice [16.063735487844628]
This study explores how artists interact with advanced text-to-image Generative AI models.
To exemplify this framework, a case study titled "Equivalence" converts users' speech input into continuously evolving paintings.
This work aims to broaden our understanding of artists' roles and foster a deeper appreciation for the creative aspects inherent in artwork created with Image Generative AI.
arXiv Detail & Related papers (2024-04-29T02:45:23Z) - Generating Human-Centric Visual Cues for Human-Object Interaction
Detection via Large Vision-Language Models [59.611697856666304]
Human-object interaction (HOI) detection aims at detecting human-object pairs and predicting their interactions.
We propose three prompts with VLM to generate human-centric visual cues within an image from multiple perspectives of humans.
We develop a transformer-based multimodal fusion module with multitower architecture to integrate visual cue features into the instance and interaction decoders.
arXiv Detail & Related papers (2023-11-26T09:11:32Z) - Detecting Any Human-Object Interaction Relationship: Universal HOI
Detector with Spatial Prompt Learning on Foundation Models [55.20626448358655]
This study explores the universal interaction recognition in an open-world setting through the use of Vision-Language (VL) foundation models and large language models (LLMs)
Our design includes an HO Prompt-guided Decoder (HOPD), facilitates the association of high-level relation representations in the foundation model with various HO pairs within the image.
For open-category interaction recognition, our method supports either of two input types: interaction phrase or interpretive sentence.
arXiv Detail & Related papers (2023-11-07T08:27:32Z) - Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation [19.62178304006683]
We argue that current interaction paradigms fall short, guiding users towards rapid convergence on a limited set of ideas.
We propose a framework that facilitates the structured generation of design space in which users can seamlessly explore, evaluate, and synthesize a multitude of responses.
arXiv Detail & Related papers (2023-10-19T17:53:14Z) - Interactive Neural Painting [66.9376011879115]
This paper proposes the first approach for Interactive Neural Painting (NP)
We propose I-Paint, a novel method based on a conditional transformer Variational AutoEncoder (VAE) architecture with a two-stage decoder.
Our experiments show that our approach provides good stroke suggestions and compares favorably to the state of the art.
arXiv Detail & Related papers (2023-07-31T07:02:00Z) - "It Felt Like Having a Second Mind": Investigating Human-AI
Co-creativity in Prewriting with Large Language Models [20.509651636971864]
This study investigates human-LLM collaboration patterns and dynamics during prewriting.
During collaborative prewriting, there appears to be a three-stage iterative Human-AI Co-creativity process.
arXiv Detail & Related papers (2023-07-20T16:55:25Z) - Inspire creativity with ORIBA: Transform Artists' Original Characters
into Chatbots through Large Language Model [4.984601297028257]
This research delves into the intersection of illustration art and artificial intelligence (AI)
By examining the impact of AI on the creative process and the boundaries of authorship, we aim to enhance human-AI interactions in creative fields.
arXiv Detail & Related papers (2023-06-16T11:25:44Z) - MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks [59.09343552273045]
We propose a decoder-only model for multimodal tasks, which is surprisingly effective in jointly learning of these disparate vision-language tasks.
We demonstrate that joint learning of these diverse objectives is simple, effective, and maximizes the weight-sharing of the model across these tasks.
Our model achieves the state of the art on image-text and text-image retrieval, video question answering and open-vocabulary detection tasks, outperforming much larger and more extensively trained foundational models.
arXiv Detail & Related papers (2023-03-29T16:42:30Z) - Pathway to Future Symbiotic Creativity [76.20798455931603]
We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist to a Machine artist in its own right.
In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations.
We propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle.
arXiv Detail & Related papers (2022-08-18T15:12:02Z) - UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes [91.24112204588353]
We introduce UViM, a unified approach capable of modeling a wide range of computer vision tasks.
In contrast to previous models, UViM has the same functional form for all tasks.
We demonstrate the effectiveness of UViM on three diverse and challenging vision tasks.
arXiv Detail & Related papers (2022-05-20T17:47:59Z) - Understanding and Creating Art with AI: Review and Outlook [12.614901374282868]
Technologies related to artificial intelligence (AI) have a strong impact on the changes of research and creative practices in visual arts.
This paper provides an integrated review of two facets of AI and art: 1) AI is used for art analysis and employed on digitized artwork collections; 2) AI is used for creative purposes and generating novel artworks.
In relation to the role of AI in creating art, we address various practical and theoretical aspects of AI Art and consolidate related works that deal with those topics in detail.
arXiv Detail & Related papers (2021-02-18T01:38:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.