Asymmetric Image Retrieval with Cross Model Compatible Ensembles
- URL: http://arxiv.org/abs/2303.17531v2
- Date: Sun, 29 Oct 2023 15:59:00 GMT
- Title: Asymmetric Image Retrieval with Cross Model Compatible Ensembles
- Authors: Ori Linial, Alon Shoshan, Nadav Bhonker, Elad Hirsch, Lior Zamir, Igor
Kviatkovsky, Gerard Medioni
- Abstract summary: asymmetrical retrieval is a well suited solution for resource constrained applications such as face recognition and image retrieval.
We present an approach that does not rely on knowledge distillation, rather it utilizes embedding transformation models.
We improve the overall accuracy beyond that of any single model while maintaining a low computational budget for querying.
- Score: 4.86935886318034
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: The asymmetrical retrieval setting is a well suited solution for resource
constrained applications such as face recognition and image retrieval. In this
setting, a large model is used for indexing the gallery while a lightweight
model is used for querying. The key principle in such systems is ensuring that
both models share the same embedding space. Most methods in this domain are
based on knowledge distillation. While useful, they suffer from several
drawbacks: they are upper-bounded by the performance of the single best model
found and cannot be extended to use an ensemble of models in a straightforward
manner. In this paper we present an approach that does not rely on knowledge
distillation, rather it utilizes embedding transformation models. This allows
the use of N independently trained and diverse gallery models (e.g., trained on
different datasets or having a different architecture) and a single query
model. As a result, we improve the overall accuracy beyond that of any single
model while maintaining a low computational budget for querying. Additionally,
we propose a gallery image rejection method that utilizes the diversity between
multiple transformed embeddings to estimate the uncertainty of gallery images.
Related papers
- A Simple Approach to Unifying Diffusion-based Conditional Generation [63.389616350290595]
We introduce a simple, unified framework to handle diverse conditional generation tasks.
Our approach enables versatile capabilities via different inference-time sampling schemes.
Our model supports additional capabilities like non-spatially aligned and coarse conditioning.
arXiv Detail & Related papers (2024-10-15T09:41:43Z) - AMES: Asymmetric and Memory-Efficient Similarity Estimation for Instance-level Retrieval [14.009257997448634]
This work investigates the problem of instance-level image retrieval re-ranking with the constraint of memory efficiency.
The proposed model uses a transformer-based architecture designed to estimate image-to-image similarity.
Results on standard benchmarks demonstrate the superiority of our approach over both hand-crafted and learned models.
arXiv Detail & Related papers (2024-08-06T16:29:51Z) - Image Similarity using An Ensemble of Context-Sensitive Models [2.9490616593440317]
We present a more intuitive approach to build and compare image similarity models based on labelled data.
We address the challenges of sparse sampling in the image space (R, A, B) and biases in the models trained with context-based data.
Our testing results show that the ensemble model constructed performs 5% better than the best individual context-sensitive models.
arXiv Detail & Related papers (2024-01-15T20:23:05Z) - Heterogeneous Federated Learning Using Knowledge Codistillation [23.895665011884102]
We propose a method that involves training a small model on the entire pool and a larger model on a subset of clients with higher capacity.
The models exchange information bidirectionally via knowledge distillation, utilizing an unlabeled dataset on a server without sharing parameters.
arXiv Detail & Related papers (2023-10-04T03:17:26Z) - FastFill: Efficient Compatible Model Update [40.27741553705222]
FastFill is a compatible model update process using feature alignment and policy based partial backfilling.
We show that previous backfilling strategies suffer from decreased performance and demonstrate the importance of both the training objective and the ordering in online partial backfilling.
arXiv Detail & Related papers (2023-03-08T18:03:51Z) - InvGAN: Invertible GANs [88.58338626299837]
InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model.
This allows us to perform image inpainting, merging, and online data augmentation.
arXiv Detail & Related papers (2021-12-08T21:39:00Z) - Meta Internal Learning [88.68276505511922]
Internal learning for single-image generation is a framework, where a generator is trained to produce novel images based on a single image.
We propose a meta-learning approach that enables training over a collection of images, in order to model the internal statistics of the sample image more effectively.
Our results show that the models obtained are as suitable as single-image GANs for many common image applications.
arXiv Detail & Related papers (2021-10-06T16:27:38Z) - Compatibility-aware Heterogeneous Visual Search [93.90831195353333]
Existing systems use the same embedding model to compute representations (embeddings) for the query and gallery images.
We address two forms of compatibility: One enforced by modifying the parameters of each model that computes the embeddings, the other by modifying the architectures that compute the embeddings.
Compared to ordinary (homogeneous) visual search using the largest embedding model (paragon), CMP-NAS achieves 80-fold and 23-fold cost reduction.
arXiv Detail & Related papers (2021-05-13T02:30:50Z) - Scale-Localized Abstract Reasoning [79.00011351374869]
We consider the abstract relational reasoning task, which is commonly used as an intelligence test.
Since some patterns have spatial rationales, while others are only semantic, we propose a multi-scale architecture that processes each query in multiple resolutions.
We show that indeed different rules are solved by different resolutions and a combined multi-scale approach outperforms the existing state of the art in this task on all benchmarks by 5-54%.
arXiv Detail & Related papers (2020-09-20T10:37:29Z) - Diverse Image Generation via Self-Conditioned GANs [56.91974064348137]
We train a class-conditional GAN model without using manually annotated class labels.
Instead, our model is conditional on labels automatically derived from clustering in the discriminator's feature space.
Our clustering step automatically discovers diverse modes, and explicitly requires the generator to cover them.
arXiv Detail & Related papers (2020-06-18T17:56:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.