Understanding Compositional Structures in Art Historical Images using
Pose and Gaze Priors
- URL: http://arxiv.org/abs/2009.03807v1
- Date: Tue, 8 Sep 2020 15:01:56 GMT
- Title: Understanding Compositional Structures in Art Historical Images using
Pose and Gaze Priors
- Authors: Prathmesh Madhu, Tilman Marquart, Ronak Kosti, Peter Bell, Andreas
Maier and Vincent Christlein
- Abstract summary: Image compositions are useful in analyzing the interactions in an image to study artists and their artworks.
In this work, we attempt to automate this process using the existing state of the art machine learning techniques.
Our approach focuses on two central themes of image composition: (a) detection of action regions and action lines of the artwork; and (b) pose-based segmentation of foreground and background.
- Score: 20.98603643788824
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image compositions as a tool for analysis of artworks is of extreme
significance for art historians. These compositions are useful in analyzing the
interactions in an image to study artists and their artworks. Max Imdahl in his
work called Ikonik, along with other prominent art historians of the 20th
century, underlined the aesthetic and semantic importance of the structural
composition of an image. Understanding underlying compositional structures
within images is challenging and a time consuming task. Generating these
structures automatically using computer vision techniques (1) can help art
historians towards their sophisticated analysis by saving lot of time;
providing an overview and access to huge image repositories and (2) also
provide an important step towards an understanding of man made imagery by
machines. In this work, we attempt to automate this process using the existing
state of the art machine learning techniques, without involving any form of
training. Our approach, inspired by Max Imdahl's pioneering work, focuses on
two central themes of image composition: (a) detection of action regions and
action lines of the artwork; and (b) pose-based segmentation of foreground and
background. Currently, our approach works for artworks comprising of
protagonists (persons) in an image. In order to validate our approach
qualitatively and quantitatively, we conduct a user study involving experts and
non-experts. The outcome of the study highly correlates with our approach and
also demonstrates its domain-agnostic capability. We have open-sourced the code
at https://github.com/image-compostion-canvas-group/image-compostion-canvas.
Related papers
- GalleryGPT: Analyzing Paintings with Large Multimodal Models [64.98398357569765]
Artwork analysis is important and fundamental skill for art appreciation, which could enrich personal aesthetic sensibility and facilitate the critical thinking ability.
Previous works for automatically analyzing artworks mainly focus on classification, retrieval, and other simple tasks, which is far from the goal of AI.
We introduce a superior large multimodal model for painting analysis composing, dubbed GalleryGPT, which is slightly modified and fine-tuned based on LLaVA architecture.
arXiv Detail & Related papers (2024-08-01T11:52:56Z) - Learning to Evaluate the Artness of AI-generated Images [64.48229009396186]
ArtScore is a metric designed to evaluate the degree to which an image resembles authentic artworks by artists.
We employ pre-trained models for photo and artwork generation, resulting in a series of mixed models.
This dataset is then employed to train a neural network that learns to estimate quantized artness levels of arbitrary images.
arXiv Detail & Related papers (2023-05-08T17:58:27Z) - Deep Image Matting: A Comprehensive Survey [85.77905619102802]
This paper presents a review of recent advancements in image matting in the era of deep learning.
We focus on two fundamental sub-tasks: auxiliary input-based image matting and automatic image matting.
We discuss relevant applications of image matting and highlight existing challenges and potential opportunities for future research.
arXiv Detail & Related papers (2023-04-10T15:48:55Z) - A domain adaptive deep learning solution for scanpath prediction of
paintings [66.46953851227454]
This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings.
We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans.
The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
arXiv Detail & Related papers (2022-09-22T22:27:08Z) - ICC++: Explainable Image Retrieval for Art Historical Corpora using
Image Composition Canvas [19.80532568090711]
We present a novel approach called Image Composition Canvas (ICC++) to compare and retrieve images having similar compositional elements.
ICC++ is an improvement over ICC specializing in generating low and high-level features (compositional elements) motivated by Max Imdahl's work.
arXiv Detail & Related papers (2022-06-22T14:06:29Z) - Automatic Image Content Extraction: Operationalizing Machine Learning in
Humanistic Photographic Studies of Large Visual Archives [81.88384269259706]
We introduce Automatic Image Content Extraction framework for machine learning-based search and analysis of large image archives.
The proposed framework can be applied in several domains in humanities and social sciences.
arXiv Detail & Related papers (2022-04-05T12:19:24Z) - Learning Portrait Style Representations [34.59633886057044]
We study style representations learned by neural network architectures incorporating higher level characteristics.
We find variation in learned style features from incorporating triplets annotated by art historians as supervision for style similarity.
We also present the first large-scale dataset of portraits prepared for computational analysis.
arXiv Detail & Related papers (2020-12-08T01:36:45Z) - A Data Set and a Convolutional Model for Iconography Classification in
Paintings [3.4138918206057265]
Iconography in art is the discipline that studies the visual content of artworks to determine their motifs and themes.
Applying Computer Vision techniques to the analysis of art images at an unprecedented scale may support iconography research and education.
arXiv Detail & Related papers (2020-10-06T12:40:46Z) - Visual link retrieval and knowledge discovery in painting datasets [14.149494915144322]
This paper presents a framework for visual link retrieval and knowledge discovery in digital painting datasets.
Visual link retrieval is accomplished by using a deep convolutional neural network to perform feature extraction.
Historical knowledge discovery is achieved by performing a graph analysis.
arXiv Detail & Related papers (2020-03-18T21:16:33Z) - Scene Text Synthesis for Efficient and Effective Deep Network Training [62.631176120557136]
We develop an innovative image synthesis technique that composes annotated training images by embedding foreground objects of interest into background images.
The proposed technique consists of two key components that in principle boost the usefulness of the synthesized images in deep network training.
Experiments over a number of public datasets demonstrate the effectiveness of our proposed image synthesis technique.
arXiv Detail & Related papers (2019-01-26T10:15:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.