Towards Improved and Interpretable Deep Metric Learning via Attentive Grouping

• URL: http://arxiv.org/abs/2011.08877v3
• Date: Wed, 25 Aug 2021 05:42:12 GMT
• Title: Towards Improved and Interpretable Deep Metric Learning via Attentive Grouping
• Authors: Xinyi Xu, Zhengyang Wang, Cheng Deng, Hao Yuan, and Shuiwang Ji
• Abstract summary: Grouping has been commonly used in deep metric learning for computing diverse features. We propose an improved and interpretable grouping method to be integrated flexibly with any metric learning framework.
• Score: 103.71992720794421
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Grouping has been commonly used in deep metric learning for computing diverse features. However, current methods are prone to overfitting and lack interpretability. In this work, we propose an improved and interpretable grouping method to be integrated flexibly with any metric learning framework. Our method is based on the attention mechanism with a learnable query for each group. The query is fully trainable and can capture group-specific information when combined with the diversity loss. An appealing property of our method is that it naturally lends itself interpretability. The attention scores between the learnable query and each spatial position can be interpreted as the importance of that position. We formally show that our proposed grouping method is invariant to spatial permutations of features. When used as a module in convolutional neural networks, our method leads to translational invariance. We conduct comprehensive experiments to evaluate our method. Our quantitative results indicate that the proposed method outperforms prior methods consistently and significantly across different datasets, evaluation metrics, base models, and loss functions. For the first time to the best of our knowledge, our interpretation results clearly demonstrate that the proposed method enables the learning of distinct and diverse features across groups. The code is available on https://github.com/XinyiXuXD/DGML-master.

Related papers

• Achieving More with Less: A Tensor-Optimization-Powered Ensemble Method [53.170053108447455]
  Ensemble learning is a method that leverages weak learners to produce a strong learner. We design a smooth and convex objective function that leverages the concept of margin, making the strong learner more discriminative. We then compare our algorithm with random forests of ten times the size and other classical methods across numerous datasets.
  arXiv  Detail & Related papers  (2024-08-06T03:42:38Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Finding Significant Features for Few-Shot Learning using Dimensionality Reduction [0.0]
  This module helps to improve the accuracy performance by allowing the similarity function, given by the metric learning method, to have more discriminative features for the classification. Our method outperforms the metric learning baselines in the miniImageNet dataset by around 2% in accuracy performance.
  arXiv  Detail & Related papers  (2021-07-06T16:36:57Z)
• Meta-learning representations for clustering with infinite Gaussian mixture models [39.56814839510978]
  We propose a meta-learning method that train neural networks for obtaining representations such that clustering performance improves. The proposed method can cluster unseen unlabeled data using knowledge meta-learned with labeled data that are different from the unlabeled data.
  arXiv  Detail & Related papers  (2021-03-01T02:05:31Z)
• Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
  We present a general methodology for interpretable evaluation for the named entity recognition (NER) task. The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them. By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
  arXiv  Detail & Related papers  (2020-11-13T10:53:27Z)
• Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
  We introduce a fast optimization-based meta-learning method for few-shot classification. Our strategy enables important aspects of the base learner objective to be learned during meta-training. We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-10-01T15:59:31Z)
• Spherical Feature Transform for Deep Metric Learning [58.35971328774927]
  This work proposes a novel spherical feature transform approach. It relaxes the assumption of identical covariance between classes to an assumption of similar covariances of different classes on a hypersphere. We provide a simple and effective training method, and in depth analysis on the relation between the two different transforms.
  arXiv  Detail & Related papers  (2020-08-04T11:32:23Z)
• Inverse Feature Learning: Feature learning based on Representation Learning of Error [12.777440204911022]
  This paper proposes inverse feature learning as a novel supervised feature learning technique that learns a set of high-level features for classification based on an error representation approach. The proposed method results in significantly better performance compared to the state-of-the-art classification techniques for several popular data sets.
  arXiv  Detail & Related papers  (2020-03-08T00:22:26Z)
• Heterogeneous Transfer Learning in Ensemble Clustering [0.0]
  We consider a clustering problem in which "similar" labeled data are available. The method is based on constructing meta-features which describe structural characteristics of data. An experimental study of the method using Monte Carlo modeling has confirmed its efficiency.
  arXiv  Detail & Related papers  (2020-01-20T16:03:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.