Insights From A Large-Scale Database of Material Depictions In Paintings

• URL: http://arxiv.org/abs/2011.12276v1
• Date: Tue, 24 Nov 2020 18:42:58 GMT
• Title: Insights From A Large-Scale Database of Material Depictions In Paintings
• Authors: Hubert Lin, Mitchell Van Zuijlen, Maarten W.A. Wijntjes, Sylvia C. Pont, Kavita Bala
• Abstract summary: We examine the give-and-take relationship between visual recognition systems and the rich information available in the fine arts. We find that visual recognition systems designed for natural images can work surprisingly well on paintings. We show that learning from paintings can be beneficial for neural networks that are intended to be used on natural images.
• Score: 18.2193253052961
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning has paved the way for strong recognition systems which are often both trained on and applied to natural images. In this paper, we examine the give-and-take relationship between such visual recognition systems and the rich information available in the fine arts. First, we find that visual recognition systems designed for natural images can work surprisingly well on paintings. In particular, we find that interactive segmentation tools can be used to cleanly annotate polygonal segments within paintings, a task which is time consuming to undertake by hand. We also find that FasterRCNN, a model which has been designed for object recognition in natural scenes, can be quickly repurposed for detection of materials in paintings. Second, we show that learning from paintings can be beneficial for neural networks that are intended to be used on natural images. We find that training on paintings instead of natural images can improve the quality of learned features and we further find that a large number of paintings can be a valuable source of test data for evaluating domain adaptation algorithms. Our experiments are based on a novel large-scale annotated database of material depictions in paintings which we detail in a separate manuscript.

Related papers

• KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities [93.74881034001312]
  We conduct a systematic study on the fidelity of entities in text-to-image generation models. We focus on their ability to generate a wide range of real-world visual entities, such as landmark buildings, aircraft, plants, and animals. Our findings reveal that even the most advanced text-to-image models often fail to generate entities with accurate visual details.
  arXiv  Detail & Related papers  (2024-10-15T17:50:37Z)
• Detecting Generated Images by Real Images Only [64.12501227493765]
  Existing generated image detection methods detect visual artifacts in generated images or learn discriminative features from both real and generated images by massive training. This paper approaches the generated image detection problem from a new perspective: Start from real images. By finding the commonality of real images and mapping them to a dense subspace in feature space, the goal is that generated images, regardless of their generative model, are then projected outside the subspace.
  arXiv  Detail & Related papers  (2023-11-02T03:09:37Z)
• Synergy of Machine and Deep Learning Models for Multi-Painter Recognition [0.0]
  We introduce a new large dataset for painting recognition task including 62 artists achieving good results. RegNet performs better in exporting features, while SVM makes the best classification of images based on the painter with a performance of up to 85%.
  arXiv  Detail & Related papers  (2023-04-28T11:34:53Z)
• Leveraging Computer Vision Application in Visual Arts: A Case Study on the Use of Residual Neural Network to Classify and Analyze Baroque Paintings [0.0]
  In this case study, we focus on the classification of a selected painting 'Portrait of the Painter Charles Bruni' by Johann Kupetzky. We show that the features extracted during residual network training can be useful for image retrieval within search systems in online art collections.
  arXiv  Detail & Related papers  (2022-10-27T10:15:36Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• One-shot Scene Graph Generation [130.57405850346836]
  We propose Multiple Structured Knowledge (Relational Knowledgesense Knowledge) for the one-shot scene graph generation task. Our method significantly outperforms existing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-02-22T11:32:59Z)
• The Curious Layperson: Fine-Grained Image Recognition without Expert Labels [90.88501867321573]
  We consider a new problem: fine-grained image recognition without expert annotations. We learn a model to describe the visual appearance of objects using non-expert image descriptions. We then train a fine-grained textual similarity model that matches image descriptions with documents on a sentence-level basis.
  arXiv  Detail & Related papers  (2021-11-05T17:58:37Z)
• SketchEmbedNet: Learning Novel Concepts by Imitating Drawings [125.45799722437478]
  We explore properties of image representations learned by training a model to produce sketches of images. We show that this generative, class-agnostic model produces informative embeddings of images from novel examples, classes, and even novel datasets in a few-shot setting.
  arXiv  Detail & Related papers  (2020-08-27T16:43:28Z)
• Diagram Image Retrieval using Sketch-Based Deep Learning and Transfer Learning [28.31395148295604]
  This paper presents a deep learning based method for image-based search for binary patent images. We begin by using deep learning to generate sketches from natural images for image retrieval. We then use our small set of manually labeled patent diagram images via transfer learning to adapt the image search from sketches of natural images to diagrams.
  arXiv  Detail & Related papers  (2020-04-22T18:27:46Z)
• Visual link retrieval and knowledge discovery in painting datasets [14.149494915144322]
  This paper presents a framework for visual link retrieval and knowledge discovery in digital painting datasets. Visual link retrieval is accomplished by using a deep convolutional neural network to perform feature extraction. Historical knowledge discovery is achieved by performing a graph analysis.
  arXiv  Detail & Related papers  (2020-03-18T21:16:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.