Data-Free Sketch-Based Image Retrieval

• URL: http://arxiv.org/abs/2303.07775v1
• Date: Tue, 14 Mar 2023 10:34:07 GMT
• Title: Data-Free Sketch-Based Image Retrieval
• Authors: Abhra Chaudhuri, Ayan Kumar Bhunia, Yi-Zhe Song, Anjan Dutta
• Abstract summary: We propose Data-Free (DF)-SBIR, where pre-trained, single-modality classification models have to be leveraged to learn cross-modal metric-space for retrieval without access to any training data. We present a methodology for DF-SBIR, which can leverage knowledge from models independently trained to perform classification on photos and sketches. Our method also achieves mAPs competitive with data-dependent approaches, all the while requiring no training data.
• Score: 56.96186184599313
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rising concerns about privacy and anonymity preservation of deep learning models have facilitated research in data-free learning (DFL). For the first time, we identify that for data-scarce tasks like Sketch-Based Image Retrieval (SBIR), where the difficulty in acquiring paired photos and hand-drawn sketches limits data-dependent cross-modal learning algorithms, DFL can prove to be a much more practical paradigm. We thus propose Data-Free (DF)-SBIR, where, unlike existing DFL problems, pre-trained, single-modality classification models have to be leveraged to learn a cross-modal metric-space for retrieval without access to any training data. The widespread availability of pre-trained classification models, along with the difficulty in acquiring paired photo-sketch datasets for SBIR justify the practicality of this setting. We present a methodology for DF-SBIR, which can leverage knowledge from models independently trained to perform classification on photos and sketches. We evaluate our model on the Sketchy, TU-Berlin, and QuickDraw benchmarks, designing a variety of baselines based on state-of-the-art DFL literature, and observe that our method surpasses all of them by significant margins. Our method also achieves mAPs competitive with data-dependent approaches, all the while requiring no training data. Implementation is available at \url{https://github.com/abhrac/data-free-sbir}.

Related papers

• One-Shot Unlearning of Personal Identities [38.36863497458095]
  One-Shot Unlearning of Personal Identities (O-UPI) evaluates unlearning models when the training data is not accessible. We benchmark the forgetting on CelebA and CelebA-HQ datasets with different unlearning set sizes. Our findings indicate that existing approaches struggle when data availability is limited, with greater difficulty when there is dissimilarity between provided samples and data used at training time.
  arXiv  Detail & Related papers  (2024-07-16T10:00:54Z)
• Data Adaptive Traceback for Vision-Language Foundation Models in Image Classification [34.37262622415682]
  We propose a new adaptation framework called Data Adaptive Traceback. Specifically, we utilize a zero-shot-based method to extract the most downstream task-related subset of the pre-training data. We adopt a pseudo-label-based semi-supervised technique to reuse the pre-training images and a vision-language contrastive learning method to address the confirmation bias issue in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-07-11T18:01:58Z)
• Modality-Aware Representation Learning for Zero-shot Sketch-based Image Retrieval [10.568851068989973]
  Zero-shot learning offers an efficient solution for a machine learning model to treat unseen categories. We propose a novel framework that indirectly aligns sketches and photos by contrasting them through texts. With an explicit modality encoding learned from data, our approach disentangles modality-agnostic semantics from modality-specific information.
  arXiv  Detail & Related papers  (2024-01-10T00:39:03Z)
• Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based Image Retrieval [69.46139774646308]
  This paper studies the problem of zero-shot sketch-based image retrieval (ZS-SBIR) It aims to use sketches from unseen categories as queries to match the images of the same category. We propose a novel Symmetrical Bidirectional Knowledge Alignment for zero-shot sketch-based image retrieval (SBKA)
  arXiv  Detail & Related papers  (2023-12-16T04:50:34Z)
• The Journey, Not the Destination: How Data Guides Diffusion Models [75.19694584942623]
  Diffusion models trained on large datasets can synthesize photo-realistic images of remarkable quality and diversity. We propose a framework that: (i) provides a formal notion of data attribution in the context of diffusion models, and (ii) allows us to counterfactually validate such attributions.
  arXiv  Detail & Related papers  (2023-12-11T08:39:43Z)
• Exploring Data Redundancy in Real-world Image Classification through Data Selection [20.389636181891515]
  Deep learning models often require large amounts of data for training, leading to increased costs. We present two data valuation metrics based on Synaptic Intelligence and gradient norms, respectively, to study redundancy in real-world image data. Online and offline data selection algorithms are then proposed via clustering and grouping based on the examined data values.
  arXiv  Detail & Related papers  (2023-06-25T03:31:05Z)
• More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval [112.1756171062067]
  We introduce a novel semi-supervised framework for cross-modal retrieval. At the centre of our design is a sequential photo-to-sketch generation model. We also introduce a discriminator guided mechanism to guide against unfaithful generation.
  arXiv  Detail & Related papers  (2021-03-25T17:27:08Z)
• Unsupervised Noisy Tracklet Person Re-identification [100.85530419892333]
  We present a novel selective tracklet learning (STL) approach that can train discriminative person re-id models from unlabelled tracklet data. This avoids the tedious and costly process of exhaustively labelling person image/tracklet true matching pairs across camera views. Our method is particularly more robust against arbitrary noisy data of raw tracklets therefore scalable to learning discriminative models from unconstrained tracking data.
  arXiv  Detail & Related papers  (2021-01-16T07:31:00Z)
• Knowledge as Priors: Cross-Modal Knowledge Generalization for Datasets without Superior Knowledge [55.32035138692167]
  Cross-modal knowledge distillation deals with transferring knowledge from a model trained with superior modalities to another model trained with weak modalities. We propose a novel scheme to train the Student in a Target dataset where the Teacher is unavailable.
  arXiv  Detail & Related papers  (2020-04-01T00:28:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.