SniffyArt: The Dataset of Smelling Persons

• URL: http://arxiv.org/abs/2311.11888v1
• Date: Mon, 20 Nov 2023 16:21:37 GMT
• Title: SniffyArt: The Dataset of Smelling Persons
• Authors: Mathias Zinnen, Azhar Hussian, Hang Tran, Prathmesh Madhu, Andreas Maier, Vincent Christlein
• Abstract summary: This paper introduces the SniffyArt dataset, consisting of 1941 individuals represented in 441 historical artworks. The dataset enables the development of hybrid classification approaches for smell gesture recognition.
• Score: 6.088288090431471
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Smell gestures play a crucial role in the investigation of past smells in the visual arts yet their automated recognition poses significant challenges. This paper introduces the SniffyArt dataset, consisting of 1941 individuals represented in 441 historical artworks. Each person is annotated with a tightly fitting bounding box, 17 pose keypoints, and a gesture label. By integrating these annotations, the dataset enables the development of hybrid classification approaches for smell gesture recognition. The datasets high-quality human pose estimation keypoints are achieved through the merging of five separate sets of keypoint annotations per person. The paper also presents a baseline analysis, evaluating the performance of representative algorithms for detection, keypoint estimation, and classification tasks, showcasing the potential of combining keypoint estimation with smell gesture classification. The SniffyArt dataset lays a solid foundation for future research and the exploration of multi-task approaches leveraging pose keypoints and person boxes to advance human gesture and olfactory dimension analysis in historical artworks.

Related papers

• Unlocking Comics: The AI4VA Dataset for Visual Understanding [62.345344799258804]
  This paper presents a novel dataset comprising Franco-Belgian comics from the 1950s annotated for tasks including depth estimation, semantic segmentation, saliency detection, and character identification. It consists of two distinct and consistent styles and incorporates object concepts and labels taken from natural images. By including such diverse information across styles, this dataset not only holds promise for computational creativity but also offers avenues for the digitization of art and storytelling innovation.
  arXiv  Detail & Related papers  (2024-10-27T14:27:05Z)
• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History [0.6345523830122167]
  We introduce the first openly licensed data set for estimating human poses in art. The Poses of People in Art data set consists of 2,454 images from 22 art-historical depiction styles. A total of 10,749 human figures are precisely enclosed by rectangular bounding boxes, with a maximum of four per image labeled by up to 17 keypoints.
  arXiv  Detail & Related papers  (2023-01-12T16:23:58Z)
• Semi-supervised Human Pose Estimation in Art-historical Images [9.633949256082763]
  We propose a novel approach to estimate human poses in art-language images. Our approach achieves significantly better results than methods that use pre-trained models or style transfer.
  arXiv  Detail & Related papers  (2022-07-06T21:20:58Z)
• PGGANet: Pose Guided Graph Attention Network for Person Re-identification [0.0]
  Person re-identification (ReID) aims at retrieving a person from images captured by different cameras. It has been proved that using local features together with global feature of person image could help to give robust feature representations for person retrieval. We propose a pose guided graph attention network, a multi-branch architecture consisting of one branch for global feature, one branch for mid-granular body features and one branch for fine-granular key point features.
  arXiv  Detail & Related papers  (2021-11-29T09:47:39Z)
• Learning Co-segmentation by Segment Swapping for Retrieval and Discovery [67.6609943904996]
  The goal of this work is to efficiently identify visually similar patterns from a pair of images. We generate synthetic training pairs by selecting object segments in an image and copy-pasting them into another image. We show our approach provides clear improvements for artwork details retrieval on the Brueghel dataset.
  arXiv  Detail & Related papers  (2021-10-29T16:51:16Z)
• Keypoint Communities [87.06615538315003]
  We present a fast bottom-up method that jointly detects over 100 keypoints on humans or objects. We use a graph centrality measure to assign training weights to different parts of a pose. Our method generalizes to car poses.
  arXiv  Detail & Related papers  (2021-10-03T11:50:34Z)
• Automatic Main Character Recognition for Photographic Studies [78.88882860340797]
  Main characters in images are the most important humans that catch the viewer's attention upon first look. Identifying the main character in images plays an important role in traditional photographic studies and media analysis. We propose a method for identifying the main characters using machine learning based human pose estimation.
  arXiv  Detail & Related papers  (2021-06-16T18:14:45Z)
• Graph-based Person Signature for Person Re-Identifications [17.181807593574764]
  We propose a new method to effectively aggregate detailed person descriptions (attributes labels) and visual features (body parts and global features) into a graph. The graph is integrated into a multi-branch multi-task framework for person re-identification. Our approach achieves competitive results among the state of the art and outperforms other attribute-based or mask-guided methods.
  arXiv  Detail & Related papers  (2021-04-14T10:54:36Z)
• Pretrained equivariant features improve unsupervised landmark discovery [69.02115180674885]
  We formulate a two-step unsupervised approach that overcomes this challenge by first learning powerful pixel-based features. Our method produces state-of-the-art results in several challenging landmark detection datasets.
  arXiv  Detail & Related papers  (2021-04-07T05:42:11Z)
• Semi-supervised Keypoint Localization [12.37129078618206]
  We propose to learn simultaneously keypoint heatmaps and pose invariant keypoint representations in a semi-supervised manner. Our approach significantly outperforms previous methods on several benchmarks for human and animal body landmark localization.
  arXiv  Detail & Related papers  (2021-01-20T06:23:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.