Non-isotropy Regularization for Proxy-based Deep Metric Learning

• URL: http://arxiv.org/abs/2203.08547v1
• Date: Wed, 16 Mar 2022 11:13:20 GMT
• Title: Non-isotropy Regularization for Proxy-based Deep Metric Learning
• Authors: Karsten Roth, Oriol Vinyals, Zeynep Akata
• Abstract summary: We propose non-isotropy regularization ($mathbbNIR$) for proxy-based Deep Metric Learning. This allows us to explicitly induce a non-isotropic distribution of samples around a proxy to optimize for. Experiments highlight consistent generalization benefits of $mathbbNIR$ while achieving competitive and state-of-the-art performance.
• Score: 78.18860829585182
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Metric Learning (DML) aims to learn representation spaces on which semantic relations can simply be expressed through predefined distance metrics. Best performing approaches commonly leverage class proxies as sample stand-ins for better convergence and generalization. However, these proxy-methods solely optimize for sample-proxy distances. Given the inherent non-bijectiveness of used distance functions, this can induce locally isotropic sample distributions, leading to crucial semantic context being missed due to difficulties resolving local structures and intraclass relations between samples. To alleviate this problem, we propose non-isotropy regularization ($\mathbb{NIR}$) for proxy-based Deep Metric Learning. By leveraging Normalizing Flows, we enforce unique translatability of samples from their respective class proxies. This allows us to explicitly induce a non-isotropic distribution of samples around a proxy to optimize for. In doing so, we equip proxy-based objectives to better learn local structures. Extensive experiments highlight consistent generalization benefits of $\mathbb{NIR}$ while achieving competitive and state-of-the-art performance on the standard benchmarks CUB200-2011, Cars196 and Stanford Online Products. In addition, we find the superior convergence properties of proxy-based methods to still be retained or even improved, making $\mathbb{NIR}$ very attractive for practical usage. Code available at https://github.com/ExplainableML/NonIsotropicProxyDML.

Related papers

• Towards Improved Proxy-based Deep Metric Learning via Data-Augmented Domain Adaptation [15.254782791542329]
  We present a novel proxy-based Deep Metric Learning framework. We propose the Data-Augmented Domain Adaptation (DADA) method to adapt the domain gap between the group of samples and proxies. Our experiments on benchmarks, including the popular CUB-200-2011, show that our learning algorithm significantly improves the existing proxy losses.
  arXiv  Detail & Related papers  (2024-01-01T00:10:58Z)
• Deep Metric Learning with Soft Orthogonal Proxies [1.823505080809275]
  We propose a novel approach that introduces Soft Orthogonality (SO) constraint on proxies. Our approach leverages Data-Efficient Image Transformer (DeiT) as an encoder to extract contextual features from images along with a DML objective. Our evaluations demonstrate the superiority of our proposed approach over state-of-the-art methods by a significant margin.
  arXiv  Detail & Related papers  (2023-06-22T17:22:15Z)
• Deep Metric Learning with Chance Constraints [6.965621436414179]
  Deep metric learning (DML) aims to empirical expected loss of the pairwise intra-/inter-class proximity violations in the embedding space. We show that minimizer of proxy-based DML satisfies certain chance constraints, and that the worst case generalization-based methods can be characterized by the radius of the smallest ball around a class proxy to cover the entire domain of the corresponding class samples, suggesting multiple proxies per class helps performance.
  arXiv  Detail & Related papers  (2022-09-19T14:50:48Z)
• A Non-isotropic Probabilistic Take on Proxy-based Deep Metric Learning [49.999268109518255]
  Proxy-based Deep Metric Learning learns by embedding images close to their class representatives (proxies) In addition, proxy-based DML struggles to learn class-internal structures. We introduce non-isotropic probabilistic proxy-based DML to address both issues.
  arXiv  Detail & Related papers  (2022-07-08T09:34:57Z)
• Self-Ensembling GAN for Cross-Domain Semantic Segmentation [107.27377745720243]
  This paper proposes a self-ensembling generative adversarial network (SE-GAN) exploiting cross-domain data for semantic segmentation. In SE-GAN, a teacher network and a student network constitute a self-ensembling model for generating semantic segmentation maps, which together with a discriminator, forms a GAN. Despite its simplicity, we find SE-GAN can significantly boost the performance of adversarial training and enhance the stability of the model.
  arXiv  Detail & Related papers  (2021-12-15T09:50:25Z)
• Instance Similarity Learning for Unsupervised Feature Representation [83.31011038813459]
  We propose an instance similarity learning (ISL) method for unsupervised feature representation. We employ the Generative Adversarial Networks (GAN) to mine the underlying feature manifold. Experiments on image classification demonstrate the superiority of our method compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-05T16:42:06Z)
• How Fine-Tuning Allows for Effective Meta-Learning [50.17896588738377]
  We present a theoretical framework for analyzing representations derived from a MAML-like algorithm. We provide risk bounds on the best predictor found by fine-tuning via gradient descent, demonstrating that the algorithm can provably leverage the shared structure. This separation result underscores the benefit of fine-tuning-based methods, such as MAML, over methods with "frozen representation" objectives in few-shot learning.
  arXiv  Detail & Related papers  (2021-05-05T17:56:00Z)
• Fewer is More: A Deep Graph Metric Learning Perspective Using Fewer Proxies [65.92826041406802]
  We propose a Proxy-based deep Graph Metric Learning approach from the perspective of graph classification. Multiple global proxies are leveraged to collectively approximate the original data points for each class. We design a novel reverse label propagation algorithm, by which the neighbor relationships are adjusted according to ground-truth labels.
  arXiv  Detail & Related papers  (2020-10-26T14:52:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.