ConceptHash: Interpretable Fine-Grained Hashing via Concept Discovery

• URL: http://arxiv.org/abs/2406.08457v1
• Date: Wed, 12 Jun 2024 17:49:26 GMT
• Title: ConceptHash: Interpretable Fine-Grained Hashing via Concept Discovery
• Authors: Kam Woh Ng, Xiatian Zhu, Yi-Zhe Song, Tao Xiang,
• Abstract summary: ConceptHash is a novel method that achieves sub-code level interpretability. In ConceptHash, each sub-code corresponds to a human-understandable concept, such as an object part. We incorporate language guidance to ensure that the learned hash codes are distinguishable within fine-grained object classes.
• Score: 128.30514851911218
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing fine-grained hashing methods typically lack code interpretability as they compute hash code bits holistically using both global and local features. To address this limitation, we propose ConceptHash, a novel method that achieves sub-code level interpretability. In ConceptHash, each sub-code corresponds to a human-understandable concept, such as an object part, and these concepts are automatically discovered without human annotations. Specifically, we leverage a Vision Transformer architecture and introduce concept tokens as visual prompts, along with image patch tokens as model inputs. Each concept is then mapped to a specific sub-code at the model output, providing natural sub-code interpretability. To capture subtle visual differences among highly similar sub-categories (e.g., bird species), we incorporate language guidance to ensure that the learned hash codes are distinguishable within fine-grained object classes while maintaining semantic alignment. This approach allows us to develop hash codes that exhibit similarity within families of species while remaining distinct from species in other families. Extensive experiments on four fine-grained image retrieval benchmarks demonstrate that ConceptHash outperforms previous methods by a significant margin, offering unique sub-code interpretability as an additional benefit. Code at: https://github.com/kamwoh/concepthash.

Related papers

• Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing [97.70862116338554]
  We investigate the problem of measuring interpretability of self-supervised representations. We formulate the latter as estimating the mutual information between the representation and a space of manually labelled concepts. We use our method to evaluate a large number of self-supervised representations, ranking them by interpretability.
  arXiv  Detail & Related papers  (2022-09-07T16:18:50Z)
• Self-Distilled Hashing for Deep Image Retrieval [25.645550298697938]
  In hash-based image retrieval systems, transformed input from the original usually generates different codes. We propose a novel self-distilled hashing scheme to minimize the discrepancy while exploiting the potential of augmented data. We also introduce hash proxy-based similarity learning and binary cross entropy-based quantization loss to provide fine quality hash codes.
  arXiv  Detail & Related papers  (2021-12-16T12:01:50Z)
• PHPQ: Pyramid Hybrid Pooling Quantization for Efficient Fine-Grained Image Retrieval [68.05570413133462]
  We propose a Pyramid Hybrid Pooling Quantization (PHPQ) module to capture and preserve fine-grained semantic information from multi-level features. Experiments on two widely-used public benchmarks, CUB-200-2011 and Stanford Dogs, demonstrate that PHPQ outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-09-11T07:21:02Z)
• Representation Learning for Efficient and Effective Similarity Search and Recommendation [6.280255585012339]
  This thesis makes contributions to representation learning that improve effectiveness of hash codes through more expressive representations and a more effective similarity measure. The contributions are empirically validated on several tasks related to similarity search and recommendation.
  arXiv  Detail & Related papers  (2021-09-04T08:19:01Z)
• CIMON: Towards High-quality Hash Codes [63.37321228830102]
  We propose a new method named textbfComprehensive stextbfImilarity textbfMining and ctextbfOnsistency leartextbfNing (CIMON) First, we use global refinement and similarity statistical distribution to obtain reliable and smooth guidance. Second, both semantic and contrastive consistency learning are introduced to derive both disturb-invariant and discriminative hash codes.
  arXiv  Detail & Related papers  (2020-10-15T14:47:14Z)
• Deep Reinforcement Learning with Label Embedding Reward for Supervised Image Hashing [85.84690941656528]
  We introduce a novel decision-making approach for deep supervised hashing. We learn a deep Q-network with a novel label embedding reward defined by Bose-Chaudhuri-Hocquenghem codes. Our approach outperforms state-of-the-art supervised hashing methods under various code lengths.
  arXiv  Detail & Related papers  (2020-08-10T09:17:20Z)
• Unsupervised Semantic Hashing with Pairwise Reconstruction [22.641786533525245]
  We present Pairwise Reconstruction (PairRec), which is a discrete variational autoencoder based hashing model. We experimentally compare PairRec to traditional and state-of-the-art approaches, and obtain significant performance improvements in the task of document similarity search.
  arXiv  Detail & Related papers  (2020-07-01T10:54:27Z)
• Dual-level Semantic Transfer Deep Hashing for Efficient Social Image Retrieval [35.78137004253608]
  Social network stores and disseminates a tremendous amount of user shared images. Deep hashing is an efficient indexing technique to support large-scale social image retrieval. Existing methods suffer from severe semantic shortage when optimizing a large amount of deep neural network parameters. We propose a Dual-level Semantic Transfer Deep Hashing (DSTDH) method to alleviate this problem.
  arXiv  Detail & Related papers  (2020-06-10T01:03:09Z)
• Reinforcing Short-Length Hashing [61.75883795807109]
  Existing methods have poor performance in retrieval using an extremely short-length hash code. In this study, we propose a novel reinforcing short-length hashing (RSLH) In this proposed RSLH, mutual reconstruction between the hash representation and semantic labels is performed to preserve the semantic information. Experiments on three large-scale image benchmarks demonstrate the superior performance of RSLH under various short-length hashing scenarios.
  arXiv  Detail & Related papers  (2020-04-24T02:23:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.