Mean Field Theory in Deep Metric Learning

• URL: http://arxiv.org/abs/2306.15368v1
• Date: Tue, 27 Jun 2023 10:33:37 GMT
• Title: Mean Field Theory in Deep Metric Learning
• Authors: Takuya Furusawa
• Abstract summary: We develop an approach to design classification-based loss functions from pair-based ones. We derive two new loss functions, MeanFieldContrastive and MeanFieldClassWiseMultiSimilarity losses, with reduced training complexity. We extensively evaluate these derived loss functions on three image-retrieval datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we explore the application of mean field theory, a technique from statistical physics, to deep metric learning and address the high training complexity commonly associated with conventional metric learning loss functions. By adapting mean field theory for deep metric learning, we develop an approach to design classification-based loss functions from pair-based ones, which can be considered complementary to the proxy-based approach. Applying the mean field theory to two pair-based loss functions, we derive two new loss functions, MeanFieldContrastive and MeanFieldClassWiseMultiSimilarity losses, with reduced training complexity. We extensively evaluate these derived loss functions on three image-retrieval datasets and demonstrate that our loss functions outperform baseline methods in two out of the three datasets.

Related papers

• Data organization limits the predictability of binary classification [8.494815916044814]
  We show that the theoretical upper bound of binary classification performance on actual datasets can be theoretically attained. Our analysis uncovers a detailed relationship between the upper limit of performance and the level of class overlap within the binary classification data.
  arXiv  Detail & Related papers  (2024-01-30T14:16:02Z)
• Class Anchor Margin Loss for Content-Based Image Retrieval [97.81742911657497]
  We propose a novel repeller-attractor loss that falls in the metric learning paradigm, yet directly optimize for the L2 metric without the need of generating pairs. We evaluate the proposed objective in the context of few-shot and full-set training on the CBIR task, by using both convolutional and transformer architectures.
  arXiv  Detail & Related papers  (2023-06-01T12:53:10Z)
• SuSana Distancia is all you need: Enforcing class separability in metric learning via two novel distance-based loss functions for few-shot image classification [0.9236074230806579]
  We propose two loss functions which consider the importance of the embedding vectors by looking at the intra-class and inter-class distance between the few data. Our results show a significant improvement in accuracy in the miniImagenNet benchmark compared to other metric-based few-shot learning methods by a margin of 2%.
  arXiv  Detail & Related papers  (2023-05-15T23:12:09Z)
• On Interpretable Approaches to Cluster, Classify and Represent Multi-Subspace Data via Minimum Lossy Coding Length based on Rate-Distortion Theory [0.0]
  Clustering, classify and represent are three fundamental objectives of learning from high-dimensional data with intrinsic structure. This paper introduces three interpretable approaches, i.e., segmentation (clustering) via the Minimum Lossy Coding Length criterion, classification via the Minimum Incremental Coding Length criterion and representation via the Maximal Coding Rate Reduction criterion.
  arXiv  Detail & Related papers  (2023-02-21T01:15:08Z)
• A survey and taxonomy of loss functions in machine learning [60.41650195728953]
  Most state-of-the-art machine learning techniques revolve around the optimisation of loss functions. This survey aims to provide a reference of the most essential loss functions for both beginner and advanced machine learning practitioners.
  arXiv  Detail & Related papers  (2023-01-13T14:38:24Z)
• Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning [12.581217671500887]
  We propose a new meta-learning framework for learning model-agnostic loss functions via a hybrid neuro-symbolic search approach. Results show that the meta-learned loss functions discovered by the newly proposed method outperform both the cross-entropy loss and state-of-the-art loss function learning methods.
  arXiv  Detail & Related papers  (2022-09-19T10:29:01Z)
• On Modality Bias Recognition and Reduction [70.69194431713825]
  We study the modality bias problem in the context of multi-modal classification. We propose a plug-and-play loss function method, whereby the feature space for each label is adaptively learned. Our method yields remarkable performance improvements compared with the baselines.
  arXiv  Detail & Related papers  (2022-02-25T13:47:09Z)
• InverseForm: A Loss Function for Structured Boundary-Aware Segmentation [80.39674800972182]
  We present a novel boundary-aware loss term for semantic segmentation using an inverse-transformation network. This plug-in loss term complements the cross-entropy loss in capturing boundary transformations. We analyze the quantitative and qualitative effects of our loss function on three indoor and outdoor segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-04-06T18:52:45Z)
• Margin-Based Transfer Bounds for Meta Learning with Deep Feature Embedding [67.09827634481712]
  We leverage margin theory and statistical learning theory to establish three margin-based transfer bounds for meta-learning based multiclass classification (MLMC) These bounds reveal that the expected error of a given classification algorithm for a future task can be estimated with the average empirical error on a finite number of previous tasks. Experiments on three benchmarks show that these margin-based models still achieve competitive performance.
  arXiv  Detail & Related papers  (2020-12-02T23:50:51Z)
• On the Benefits of Invariance in Neural Networks [56.362579457990094]
  We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation. We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
  arXiv  Detail & Related papers  (2020-05-01T02:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.