Visual link retrieval and knowledge discovery in painting datasets

• URL: http://arxiv.org/abs/2003.08476v2
• Date: Thu, 22 Oct 2020 15:31:07 GMT
• Title: Visual link retrieval and knowledge discovery in painting datasets
• Authors: Giovanna Castellano and Eufemia Lella and Gennaro Vessio
• Abstract summary: This paper presents a framework for visual link retrieval and knowledge discovery in digital painting datasets. Visual link retrieval is accomplished by using a deep convolutional neural network to perform feature extraction. Historical knowledge discovery is achieved by performing a graph analysis.
• Score: 14.149494915144322
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual arts are of inestimable importance for the cultural, historic and economic growth of our society. One of the building blocks of most analysis in visual arts is to find similarity relationships among paintings of different artists and painting schools. To help art historians better understand visual arts, this paper presents a framework for visual link retrieval and knowledge discovery in digital painting datasets. Visual link retrieval is accomplished by using a deep convolutional neural network to perform feature extraction and a fully unsupervised nearest neighbor mechanism to retrieve links among digitized paintings. Historical knowledge discovery is achieved by performing a graph analysis that makes it possible to study influences among artists. An experimental evaluation on a database collecting paintings by very popular artists shows the effectiveness of the method. The unsupervised strategy makes the method interesting especially in cases where metadata are scarce, unavailable or difficult to collect.

Related papers

• GalleryGPT: Analyzing Paintings with Large Multimodal Models [64.98398357569765]
  Artwork analysis is important and fundamental skill for art appreciation, which could enrich personal aesthetic sensibility and facilitate the critical thinking ability. Previous works for automatically analyzing artworks mainly focus on classification, retrieval, and other simple tasks, which is far from the goal of AI. We introduce a superior large multimodal model for painting analysis composing, dubbed GalleryGPT, which is slightly modified and fine-tuned based on LLaVA architecture.
  arXiv  Detail & Related papers  (2024-08-01T11:52:56Z)
• Learning to Evaluate the Artness of AI-generated Images [64.48229009396186]
  ArtScore is a metric designed to evaluate the degree to which an image resembles authentic artworks by artists. We employ pre-trained models for photo and artwork generation, resulting in a series of mixed models. This dataset is then employed to train a neural network that learns to estimate quantized artness levels of arbitrary images.
  arXiv  Detail & Related papers  (2023-05-08T17:58:27Z)
• Synergy of Machine and Deep Learning Models for Multi-Painter Recognition [0.0]
  We introduce a new large dataset for painting recognition task including 62 artists achieving good results. RegNet performs better in exporting features, while SVM makes the best classification of images based on the painter with a performance of up to 85%.
  arXiv  Detail & Related papers  (2023-04-28T11:34:53Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• One-shot Scene Graph Generation [130.57405850346836]
  We propose Multiple Structured Knowledge (Relational Knowledgesense Knowledge) for the one-shot scene graph generation task. Our method significantly outperforms existing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-02-22T11:32:59Z)
• Compositional Scene Representation Learning via Reconstruction: A Survey [48.33349317481124]
  Compositional scene representation learning is a task that enables such abilities. Deep neural networks have been proven to be advantageous in representation learning. Learning via reconstruction is advantageous because it may utilize massive unlabeled data and avoid costly and laborious data annotation.
  arXiv  Detail & Related papers  (2022-02-15T02:14:05Z)
• Learning Co-segmentation by Segment Swapping for Retrieval and Discovery [67.6609943904996]
  The goal of this work is to efficiently identify visually similar patterns from a pair of images. We generate synthetic training pairs by selecting object segments in an image and copy-pasting them into another image. We show our approach provides clear improvements for artwork details retrieval on the Brueghel dataset.
  arXiv  Detail & Related papers  (2021-10-29T16:51:16Z)
• ArtGraph: Towards an Artistic Knowledge Graph [6.233095477694583]
  This paper presents our ongoing work towards ArtGraph: an artistic knowledge graph based on WikiArt and DBpedia. A knowledge graph that integrates a rich body of information about artworks, artists, painting schools, etc., in a unified structured framework can provide a valuable resource for more powerful information retrieval and knowledge discovery tools in the artistic domain.
  arXiv  Detail & Related papers  (2021-05-31T15:09:05Z)
• Insights From A Large-Scale Database of Material Depictions In Paintings [18.2193253052961]
  We examine the give-and-take relationship between visual recognition systems and the rich information available in the fine arts. We find that visual recognition systems designed for natural images can work surprisingly well on paintings. We show that learning from paintings can be beneficial for neural networks that are intended to be used on natural images.
  arXiv  Detail & Related papers  (2020-11-24T18:42:58Z)
• Demographic Influences on Contemporary Art with Unsupervised Style Embeddings [25.107166631583212]
  contempArt is a collection of paintings and drawings, a detailed graph network based on social connections on Instagram and additional socio-demographic information. We evaluate three methods suited for generating unsupervised style embeddings of images and correlate them with the remaining data.
  arXiv  Detail & Related papers  (2020-09-30T10:13:18Z)
• Understanding Compositional Structures in Art Historical Images using Pose and Gaze Priors [20.98603643788824]
  Image compositions are useful in analyzing the interactions in an image to study artists and their artworks. In this work, we attempt to automate this process using the existing state of the art machine learning techniques. Our approach focuses on two central themes of image composition: (a) detection of action regions and action lines of the artwork; and (b) pose-based segmentation of foreground and background.
  arXiv  Detail & Related papers  (2020-09-08T15:01:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.