Unifying Specialist Image Embedding into Universal Image Embedding
- URL: http://arxiv.org/abs/2003.03701v1
- Date: Sun, 8 Mar 2020 02:51:11 GMT
- Title: Unifying Specialist Image Embedding into Universal Image Embedding
- Authors: Yang Feng, Futang Peng, Xu Zhang, Wei Zhu, Shanfeng Zhang, Howard
Zhou, Zhen Li, Tom Duerig, Shih-Fu Chang, Jiebo Luo
- Abstract summary: It is desirable to have a universal deep embedding model applicable to various domains of images.
We propose to distill the knowledge in multiple specialists into a universal embedding to solve this problem.
- Score: 84.0039266370785
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep image embedding provides a way to measure the semantic similarity of two
images. It plays a central role in many applications such as image search, face
verification, and zero-shot learning. It is desirable to have a universal deep
embedding model applicable to various domains of images. However, existing
methods mainly rely on training specialist embedding models each of which is
applicable to images from a single domain. In this paper, we study an important
but unexplored task: how to train a single universal image embedding model to
match the performance of several specialists on each specialist's domain.
Simply fusing the training data from multiple domains cannot solve this problem
because some domains become overfitted sooner when trained together using
existing methods. Therefore, we propose to distill the knowledge in multiple
specialists into a universal embedding to solve this problem. In contrast to
existing embedding distillation methods that distill the absolute distances
between images, we transform the absolute distances between images into a
probabilistic distribution and minimize the KL-divergence between the
distributions of the specialists and the universal embedding. Using several
public datasets, we validate that our proposed method accomplishes the goal of
universal image embedding.
Related papers
- CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
We propose a robust global representation method with cross-image correlation awareness for visual place recognition.
Our method uses the attention mechanism to correlate multiple images within a batch.
Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
arXiv Detail & Related papers (2024-02-29T15:05:11Z) - Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge
for Generic Image Representations [4.606379774346321]
We address the problem of universal image embedding, where a single universal model is trained and used in multiple domains.
First, we leverage existing domain-specific datasets to carefully construct a new large-scale public benchmark for the evaluation of universal image embeddings.
Second, we provide a comprehensive experimental evaluation on the new dataset, demonstrating that existing approaches and simplistic extensions lead to worse performance than an assembly of models trained for each domain separately.
arXiv Detail & Related papers (2023-09-04T23:18:38Z) - Domain Generalization for Mammographic Image Analysis with Contrastive
Learning [62.25104935889111]
The training of an efficacious deep learning model requires large data with diverse styles and qualities.
A novel contrastive learning is developed to equip the deep learning models with better style generalization capability.
The proposed method has been evaluated extensively and rigorously with mammograms from various vendor style domains and several public datasets.
arXiv Detail & Related papers (2023-04-20T11:40:21Z) - Internal Diverse Image Completion [38.068971605321096]
We propose a diverse completion method that does not require a training set and can treat arbitrary images from any domain.
Our internal diverse completion (IDC) approach draws inspiration from recent single-image generative models that are trained on multiple scales of a single image.
arXiv Detail & Related papers (2022-12-18T10:02:53Z) - Fuse and Attend: Generalized Embedding Learning for Art and Sketches [6.375982344506753]
We propose a novel Embedding Learning approach with the goal of generalizing across different domains.
We show the prowess of our method using the Domain framework, on the popular PACS (Photo, Art painting, Cartoon, and Sketch) dataset.
arXiv Detail & Related papers (2022-08-20T14:44:11Z) - WEDGE: Web-Image Assisted Domain Generalization for Semantic
Segmentation [72.88657378658549]
We propose a WEb-image assisted Domain GEneralization scheme, which is the first to exploit the diversity of web-crawled images for generalizable semantic segmentation.
We also present a method which injects styles of the web-crawled images into training images on-the-fly during training, which enables the network to experience images of diverse styles with reliable labels for effective training.
arXiv Detail & Related papers (2021-09-29T05:19:58Z) - Bridging Composite and Real: Towards End-to-end Deep Image Matting [88.79857806542006]
We study the roles of semantics and details for image matting.
We propose a novel Glance and Focus Matting network (GFM), which employs a shared encoder and two separate decoders.
Comprehensive empirical studies have demonstrated that GFM outperforms state-of-the-art methods.
arXiv Detail & Related papers (2020-10-30T10:57:13Z) - Universal Model for Multi-Domain Medical Image Retrieval [88.67940265012638]
Medical Image Retrieval (MIR) helps doctors quickly find similar patients' data.
MIR is becoming increasingly helpful due to the wide use of digital imaging modalities.
However, the popularity of various digital imaging modalities in hospitals also poses several challenges to MIR.
arXiv Detail & Related papers (2020-07-14T23:22:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.