Related papers: Context-aware Multimodal AI Reveals Hidden Pathways in Five Centuries of Art Evolution

Context-aware Multimodal AI Reveals Hidden Pathways in Five Centuries of Art Evolution

URL: http://arxiv.org/abs/2503.13531v1
Date: Sat, 15 Mar 2025 10:45:04 GMT
Title: Context-aware Multimodal AI Reveals Hidden Pathways in Five Centuries of Art Evolution
Authors: Jin Kim, Byunghwee Lee, Taekho You, Jinhyuk Yun,
Abstract summary: We use cutting-edge generative AI, specifically Stable Diffusion, to analyze 500 years of Western paintings.<n>Our findings reveal that contextual information differentiates between artistic periods, styles, and individual artists more successfully than formal elements.<n>Our generative experiment, infusing prospective contexts into historical artworks, successfully reproduces the evolutionary trajectory of artworks.
Score: 1.8435193934665342
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: The rise of multimodal generative AI is transforming the intersection of technology and art, offering deeper insights into large-scale artwork. Although its creative capabilities have been widely explored, its potential to represent artwork in latent spaces remains underexamined. We use cutting-edge generative AI, specifically Stable Diffusion, to analyze 500 years of Western paintings by extracting two types of latent information with the model: formal aspects (e.g., colors) and contextual aspects (e.g., subject). Our findings reveal that contextual information differentiates between artistic periods, styles, and individual artists more successfully than formal elements. Additionally, using contextual keywords extracted from paintings, we show how artistic expression evolves alongside societal changes. Our generative experiment, infusing prospective contexts into historical artworks, successfully reproduces the evolutionary trajectory of artworks, highlighting the significance of mutual interaction between society and art. This study demonstrates how multimodal AI expands traditional formal analysis by integrating temporal, cultural, and historical contexts.

Related papers

Diffusion-Based Visual Art Creation: A Survey and New Perspectives [51.522935314070416]
This survey explores the emerging realm of diffusion-based visual art creation, examining its development from both artistic and technical perspectives. Our findings reveal how artistic requirements are transformed into technical challenges and highlight the design and application of diffusion-based methods within visual art creation. We aim to shed light on the mechanisms through which AI systems emulate and possibly, enhance human capacities in artistic perception and creativity.
arXiv Detail & Related papers (2024-08-22T04:49:50Z)
GalleryGPT: Analyzing Paintings with Large Multimodal Models [64.98398357569765]
Artwork analysis is important and fundamental skill for art appreciation, which could enrich personal aesthetic sensibility and facilitate the critical thinking ability. Previous works for automatically analyzing artworks mainly focus on classification, retrieval, and other simple tasks, which is far from the goal of AI. We introduce a superior large multimodal model for painting analysis composing, dubbed GalleryGPT, which is slightly modified and fine-tuned based on LLaVA architecture.
arXiv Detail & Related papers (2024-08-01T11:52:56Z)
Equivalence: An analysis of artists' roles with Image Generative AI from Conceptual Art perspective through an interactive installation design practice [16.063735487844628]
This study explores how artists interact with advanced text-to-image Generative AI models. To exemplify this framework, a case study titled "Equivalence" converts users' speech input into continuously evolving paintings. This work aims to broaden our understanding of artists' roles and foster a deeper appreciation for the creative aspects inherent in artwork created with Image Generative AI.
arXiv Detail & Related papers (2024-04-29T02:45:23Z)
AI Art Neural Constellation: Revealing the Collective and Contrastive State of AI-Generated and Human Art [36.21731898719347]
We conduct a comprehensive analysis to position AI-generated art within the context of human art heritage. Our comparative analysis is based on an extensive dataset, dubbed ArtConstellation'' Key finding is that AI-generated artworks are visually related to the principle concepts for modern period art made in 1800-2000.
arXiv Detail & Related papers (2024-02-04T11:49:51Z)
CreativeSynth: Creative Blending and Synthesis of Visual Arts based on Multimodal Diffusion [74.44273919041912]
Large-scale text-to-image generative models have made impressive strides, showcasing their ability to synthesize a vast array of high-quality images. However, adapting these models for artistic image editing presents two significant challenges. We build the innovative unified framework Creative Synth, which is based on a diffusion model with the ability to coordinate multimodal inputs.
arXiv Detail & Related papers (2024-01-25T10:42:09Z)
Vision+X: A Survey on Multimodal Learning in the Light of Data [64.03266872103835]
multimodal machine learning that incorporates data from various sources has become an increasingly popular research area. We analyze the commonness and uniqueness of each data format mainly ranging from vision, audio, text, and motions. We investigate the existing literature on multimodal learning from both the representation learning and downstream application levels.
arXiv Detail & Related papers (2022-10-05T13:14:57Z)
Pathway to Future Symbiotic Creativity [76.20798455931603]
We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist to a Machine artist in its own right. In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations. We propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle.
arXiv Detail & Related papers (2022-08-18T15:12:02Z)
Art Creation with Multi-Conditional StyleGANs [81.72047414190482]
A human artist needs a combination of unique skills, understanding, and genuine intention to create artworks that evoke deep feelings and emotions. We introduce a multi-conditional Generative Adversarial Network (GAN) approach trained on large amounts of human paintings to synthesize realistic-looking paintings that emulate human art.
arXiv Detail & Related papers (2022-02-23T20:45:41Z)
Quantifying Confounding Bias in Generative Art: A Case Study [3.198144010381572]
We propose a simple metric to quantify confounding bias due to the lack of modeling the influence of art movements in learning artists' styles. The proposed metric is more effective than state-of-the-art outlier detection method in understanding the influence of art movements in artworks.
arXiv Detail & Related papers (2021-02-23T21:59:30Z)
Understanding and Creating Art with AI: Review and Outlook [12.614901374282868]
Technologies related to artificial intelligence (AI) have a strong impact on the changes of research and creative practices in visual arts. This paper provides an integrated review of two facets of AI and art: 1) AI is used for art analysis and employed on digitized artwork collections; 2) AI is used for creative purposes and generating novel artworks. In relation to the role of AI in creating art, we address various practical and theoretical aspects of AI Art and consolidate related works that deal with those topics in detail.
arXiv Detail & Related papers (2021-02-18T01:38:11Z)
Biases in Generative Art -- A Causal Look from the Lens of Art History [3.198144010381572]
We investigate biases in the generative art AI pipeline from those that can originate due to improper problem formulation to those related to algorithm design. We highlight how current methods fall short in modeling the process of art creation and thus contribute to various types of biases.
arXiv Detail & Related papers (2020-10-26T00:49:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.