A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning
- URL: http://arxiv.org/abs/2207.03784v1
- Date: Fri, 8 Jul 2022 09:34:57 GMT
- Title: A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning
- Authors: Michael Kirchhof, Karsten Roth, Zeynep Akata, Enkelejda Kasneci
- Abstract summary: Proxy-based Deep Metric Learning learns by embedding images close to their class representatives (proxies)
In addition, proxy-based DML struggles to learn class-internal structures.
We introduce non-isotropic probabilistic proxy-based DML to address both issues.
- Score: 49.999268109518255
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Proxy-based Deep Metric Learning (DML) learns deep representations by
embedding images close to their class representatives (proxies), commonly with
respect to the angle between them. However, this disregards the embedding norm,
which can carry additional beneficial context such as class- or image-intrinsic
uncertainty. In addition, proxy-based DML struggles to learn class-internal
structures. To address both issues at once, we introduce non-isotropic
probabilistic proxy-based DML. We model images as directional von Mises-Fisher
(vMF) distributions on the hypersphere that can reflect image-intrinsic
uncertainties. Further, we derive non-isotropic von Mises-Fisher (nivMF)
distributions for class proxies to better represent complex class-specific
variances. To measure the proxy-to-image distance between these models, we
develop and investigate multiple distribution-to-point and
distribution-to-distribution metrics. Each framework choice is motivated by a
set of ablational studies, which showcase beneficial properties of our
probabilistic approach to proxy-based DML, such as uncertainty-awareness,
better-behaved gradients during training, and overall improved generalization
performance. The latter is especially reflected in the competitive performance
on the standard DML benchmarks, where our approach compares favorably,
suggesting that existing proxy-based DML can significantly benefit from a more
probabilistic treatment. Code is available at
github.com/ExplainableML/Probabilistic_Deep_Metric_Learning.
Related papers
- Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode.
We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
arXiv Detail & Related papers (2024-06-05T16:35:30Z) - Towards Improved Proxy-based Deep Metric Learning via Data-Augmented
Domain Adaptation [15.254782791542329]
We present a novel proxy-based Deep Metric Learning framework.
We propose the Data-Augmented Domain Adaptation (DADA) method to adapt the domain gap between the group of samples and proxies.
Our experiments on benchmarks, including the popular CUB-200-2011, show that our learning algorithm significantly improves the existing proxy losses.
arXiv Detail & Related papers (2024-01-01T00:10:58Z) - Learning Invariant Molecular Representation in Latent Discrete Space [52.13724532622099]
We propose a new framework for learning molecular representations that exhibit invariance and robustness against distribution shifts.
Our model achieves stronger generalization against state-of-the-art baselines in the presence of various distribution shifts.
arXiv Detail & Related papers (2023-10-22T04:06:44Z) - ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models [69.50316788263433]
We propose ProbVLM, a probabilistic adapter that estimates probability distributions for the embeddings of pre-trained vision-language models.
We quantify the calibration of embedding uncertainties in retrieval tasks and show that ProbVLM outperforms other methods.
We present a novel technique for visualizing the embedding distributions using a large-scale pre-trained latent diffusion model.
arXiv Detail & Related papers (2023-07-01T18:16:06Z) - Deep Metric Learning with Soft Orthogonal Proxies [1.823505080809275]
We propose a novel approach that introduces Soft Orthogonality (SO) constraint on proxies.
Our approach leverages Data-Efficient Image Transformer (DeiT) as an encoder to extract contextual features from images along with a DML objective.
Our evaluations demonstrate the superiority of our proposed approach over state-of-the-art methods by a significant margin.
arXiv Detail & Related papers (2023-06-22T17:22:15Z) - Deep Metric Learning with Chance Constraints [6.965621436414179]
Deep metric learning (DML) aims to empirical expected loss of the pairwise intra-/inter-class proximity violations in the embedding space.
We show that minimizer of proxy-based DML satisfies certain chance constraints, and that the worst case generalization-based methods can be characterized by the radius of the smallest ball around a class proxy to cover the entire domain of the corresponding class samples, suggesting multiple proxies per class helps performance.
arXiv Detail & Related papers (2022-09-19T14:50:48Z) - Learning with Stochastic Orders [25.795107089736295]
Learning high-dimensional distributions is often done with explicit likelihood modeling or implicit modeling via integral probability metrics (IPMs)
We introduce the Choquet-Toland distance between probability measures, that can be used as a drop-in replacement for IPMsational.
We also introduce the Variational Dominance Criterion (VDC) to learn probability measures with dominance constraints.
arXiv Detail & Related papers (2022-05-27T00:08:03Z) - Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios.
We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
arXiv Detail & Related papers (2022-04-13T12:43:12Z) - Non-isotropy Regularization for Proxy-based Deep Metric Learning [78.18860829585182]
We propose non-isotropy regularization ($mathbbNIR$) for proxy-based Deep Metric Learning.
This allows us to explicitly induce a non-isotropic distribution of samples around a proxy to optimize for.
Experiments highlight consistent generalization benefits of $mathbbNIR$ while achieving competitive and state-of-the-art performance.
arXiv Detail & Related papers (2022-03-16T11:13:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.