Beyond Domain Gap: Exploiting Subjectivity in Sketch-Based Person Retrieval

• URL: http://arxiv.org/abs/2309.08372v1
• Date: Fri, 15 Sep 2023 12:59:01 GMT
• Title: Beyond Domain Gap: Exploiting Subjectivity in Sketch-Based Person Retrieval
• Authors: Kejun Lin and Zhixiang Wang and Zheng Wang and Yinqiang Zheng and Shin'ichi Satoh
• Abstract summary: Person re-identification (re-ID) requires densely distributed cameras. Previous research defines this case using the sketch as sketch re-identification (Sketch re-ID) We model and investigate it by posing a new dataset with multi-witness descriptions. It contains over 4,763 sketches and 32,668 photos, making it the largest Sketch re-ID dataset.
• Score: 40.257842079152255
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Person re-identification (re-ID) requires densely distributed cameras. In practice, the person of interest may not be captured by cameras and, therefore, needs to be retrieved using subjective information (e.g., sketches from witnesses). Previous research defines this case using the sketch as sketch re-identification (Sketch re-ID) and focuses on eliminating the domain gap. Actually, subjectivity is another significant challenge. We model and investigate it by posing a new dataset with multi-witness descriptions. It features two aspects. 1) Large-scale. It contains over 4,763 sketches and 32,668 photos, making it the largest Sketch re-ID dataset. 2) Multi-perspective and multi-style. Our dataset offers multiple sketches for each identity. Witnesses' subjective cognition provides multiple perspectives on the same individual, while different artists' drawing styles provide variation in sketch styles. We further have two novel designs to alleviate the challenge of subjectivity. 1) Fusing subjectivity. We propose a non-local (NL) fusion module that gathers sketches from different witnesses for the same identity. 2) Introducing objectivity. An AttrAlign module utilizes attributes as an implicit mask to align cross-domain features. To push forward the advance of Sketch re-ID, we set three benchmarks (large-scale, multi-style, cross-style). Extensive experiments demonstrate our leading performance in these benchmarks. Dataset and Codes are publicly available at: https://github.com/Lin-Kayla/subjectivity-sketch-reid

Related papers

• Freeview Sketching: View-Aware Fine-Grained Sketch-Based Image Retrieval [85.73149096516543]
  We address the choice of viewpoint during sketch creation in Fine-Grained Sketch-Based Image Retrieval (FG-SBIR) A pilot study highlights the system's struggle when query-sketches differ in viewpoint from target instances. To reconcile this, we advocate for a view-aware system, seamlessly accommodating both view-agnostic and view-specific tasks.
  arXiv  Detail & Related papers  (2024-07-01T21:20:44Z)
• SketchTriplet: Self-Supervised Scenarized Sketch-Text-Image Triplet Generation [6.39528707908268]
  There continues to be a lack of large-scale paired datasets for scene sketches. We propose a self-supervised method for scene sketch generation that does not rely on any existing scene sketch. We contribute a large-scale dataset centered around scene sketches, comprising highly semantically consistent "text-sketch-image" triplets.
  arXiv  Detail & Related papers  (2024-05-29T06:43:49Z)
• Fine-Tuned but Zero-Shot 3D Shape Sketch View Similarity and Retrieval [8.540349872620993]
  We show that in a zero-shot setting, the more abstract the sketch, the higher the likelihood of incorrect image matches. One of the key findings of our research is that meticulous fine-tuning on one class of 3D shapes can lead to improved performance on other shape classes.
  arXiv  Detail & Related papers  (2023-06-14T14:40:50Z)
• Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings [99.9788496281408]
  We study how sketches can be used as a weak label to detect salient objects present in an image. To accomplish this, we introduce a photo-to-sketch generation model that aims to generate sequential sketch coordinates corresponding to a given visual photo. Tests prove our hypothesis and delineate how our sketch-based saliency detection model gives a competitive performance compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2023-03-20T23:46:46Z)
• DifferSketching: How Differently Do People Sketch 3D Objects? [78.44544977215918]
  Multiple sketch datasets have been proposed to understand how people draw 3D objects. These datasets are often of small scale and cover a small set of objects or categories. We analyze the collected data at three levels, i.e., sketch-level, stroke-level, and pixel-level, under both spatial and temporal characteristics.
  arXiv  Detail & Related papers  (2022-09-19T06:52:18Z)
• On Learning Semantic Representations for Million-Scale Free-Hand Sketches [146.52892067335128]
  We study learning semantic representations for million-scale free-hand sketches. We propose a dual-branch CNNRNN network architecture to represent sketches. We explore learning the sketch-oriented semantic representations in hashing retrieval and zero-shot recognition.
  arXiv  Detail & Related papers  (2020-07-07T15:23:22Z)
• SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence [68.63311821718416]
  We study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object. This problem is challenging since the visual features of corresponding points at different views can be very different. We take a deep learning approach and learn a novel local sketch descriptor from data.
  arXiv  Detail & Related papers  (2020-01-16T11:31:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.