Multi-Class Zero-Shot Learning for Artistic Material Recognition

• URL: http://arxiv.org/abs/2010.13850v1
• Date: Mon, 26 Oct 2020 19:04:50 GMT
• Title: Multi-Class Zero-Shot Learning for Artistic Material Recognition
• Authors: Alexander W Olson, Andreea Cucu, Tom Bock
• Abstract summary: Zero-Shot Learning (ZSL) is an extreme form of transfer learning, where no labelled examples of the data to be classified are provided during the training stage. Here we outline a model to identify the materials with which a work of art was created, by learning the relationship between English descriptions of the subject of a piece and its composite materials. We produce a model which is capable of correctly identifying the materials used on pieces from an entirely distinct museum dataset.
• Score: 68.8204255655161
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-Shot Learning (ZSL) is an extreme form of transfer learning, where no labelled examples of the data to be classified are provided during the training stage. Instead, ZSL uses additional information learned about the domain, and relies upon transfer learning algorithms to infer knowledge about the missing instances. ZSL approaches are an attractive solution for sparse datasets. Here we outline a model to identify the materials with which a work of art was created, by learning the relationship between English descriptions of the subject of a piece and its composite materials. After experimenting with a range of hyper-parameters, we produce a model which is capable of correctly identifying the materials used on pieces from an entirely distinct museum dataset. This model returned a classification accuracy of 48.42% on 5,000 artworks taken from the Tate collection, which is distinct from the Rijksmuseum network used to create and train our model.

Related papers

• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• Deep Semantic-Visual Alignment for Zero-Shot Remote Sensing Image Scene Classification [26.340737217001497]
  Zero-shot learning (ZSL) allows for identifying novel classes that are not seen during training. Previous ZSL models mainly depend on manually-labeled attributes or word embeddings extracted from language models to transfer knowledge from seen classes to novel classes. We propose to collect visually detectable attributes automatically. We predict attributes for each class by depicting the semantic-visual similarity between attributes and images.
  arXiv  Detail & Related papers  (2024-02-03T09:18:49Z)
• Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection [56.292071534857946]
  Recent data-privacy laws have sparked interest in machine unlearning. Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset. We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU) We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
  arXiv  Detail & Related papers  (2023-12-07T07:17:24Z)
• NLLB-CLIP -- train performant multilingual image retrieval model on a budget [65.268245109828]
  We present NLLB-CLIP - CLIP model with a text encoder from the NLLB model. We used an automatically created dataset of 106,246 good-quality images with captions in 201 languages. We show that NLLB-CLIP is comparable in quality to state-of-the-art models and significantly outperforms them on low-resource languages.
  arXiv  Detail & Related papers  (2023-09-04T23:26:11Z)
• Recognizing Unseen Objects via Multimodal Intensive Knowledge Graph Propagation [68.13453771001522]
  We propose a multimodal intensive ZSL framework that matches regions of images with corresponding semantic embeddings. We conduct extensive experiments and evaluate our model on large-scale real-world data.
  arXiv  Detail & Related papers  (2023-06-14T13:07:48Z)
• Terrain Classification using Transfer Learning on Hyperspectral Images: A Comparative study [0.13999481573773068]
  convolutional neural network (CNN) and the Multi-Layer Perceptron (MLP) have been proven to be an effective method of image classification. However, they suffer from the issues of long training time and requirement of large amounts of the labeled data. We propose using the method of transfer learning to decrease the training time and reduce the dependence on large labeled dataset.
  arXiv  Detail & Related papers  (2022-06-19T14:36:33Z)
• Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision [38.22842778742829]
  Discriminative self-supervised learning allows training models on any random group of internet images. We train models on billions of random images without any data pre-processing or prior assumptions about what we want the model to learn. We extensively study and validate our model performance on over 50 benchmarks including fairness, to distribution shift, geographical diversity, fine grained recognition, image copy detection and many image classification datasets.
  arXiv  Detail & Related papers  (2022-02-16T22:26:47Z)
• Learning from scarce information: using synthetic data to classify Roman fine ware pottery [0.0]
  We propose to use a transfer learning approach whereby the model is first trained on a synthetic dataset replicating features of the original objects. Taking the replicated features from published profile drawings of pottery forms allowed the integration of expert knowledge into the process. After this first initial training the model was fine-tuned with data from photographs of real vessels.
  arXiv  Detail & Related papers  (2021-07-03T10:30:46Z)
• Few-Cost Salient Object Detection with Adversarial-Paced Learning [95.0220555274653]
  This paper proposes to learn the effective salient object detection model based on the manual annotation on a few training images only. We name this task as the few-cost salient object detection and propose an adversarial-paced learning (APL)-based framework to facilitate the few-cost learning scenario.
  arXiv  Detail & Related papers  (2021-04-05T14:15:49Z)
• Zero-shot Relation Classification from Side Information [5.609443065827996]
  A zero-shot learning approach mimics the way humans learn and recognize new concepts with no prior knowledge. ZSLRC uses advanced networks that are modified to utilize weighted side (auxiliary) information. ZSLRC significantly outperforms state-of-the-art methods on supervised learning, few-shot learning, and zero-shot learning tasks.
  arXiv  Detail & Related papers  (2020-11-13T20:57:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.