Circle Loss: A Unified Perspective of Pair Similarity Optimization

• URL: http://arxiv.org/abs/2002.10857v2
• Date: Mon, 15 Jun 2020 08:15:47 GMT
• Title: Circle Loss: A Unified Perspective of Pair Similarity Optimization
• Authors: Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, Yichen Wei
• Abstract summary: We find a majority of loss functions, including the triplet loss and the softmax plus cross-entropy loss. We show that the Circle loss offers a more flexible optimization approach towards a more definite convergence target.
• Score: 42.33948436767691
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper provides a pair similarity optimization viewpoint on deep feature learning, aiming to maximize the within-class similarity $s_p$ and minimize the between-class similarity $s_n$. We find a majority of loss functions, including the triplet loss and the softmax plus cross-entropy loss, embed $s_n$ and $s_p$ into similarity pairs and seek to reduce $(s_n-s_p)$. Such an optimization manner is inflexible, because the penalty strength on every single similarity score is restricted to be equal. Our intuition is that if a similarity score deviates far from the optimum, it should be emphasized. To this end, we simply re-weight each similarity to highlight the less-optimized similarity scores. It results in a Circle loss, which is named due to its circular decision boundary. The Circle loss has a unified formula for two elemental deep feature learning approaches, i.e. learning with class-level labels and pair-wise labels. Analytically, we show that the Circle loss offers a more flexible optimization approach towards a more definite convergence target, compared with the loss functions optimizing $(s_n-s_p)$. Experimentally, we demonstrate the superiority of the Circle loss on a variety of deep feature learning tasks. On face recognition, person re-identification, as well as several fine-grained image retrieval datasets, the achieved performance is on par with the state of the art.

Related papers

• Faster Convergence with Multiway Preferences [99.68922143784306]
  We consider the sign-function-based comparison feedback model and analyze the convergence rates with batched and multiway comparisons. Our work is the first to study the problem of convex optimization with multiway preferences and analyze the optimal convergence rates.
  arXiv  Detail & Related papers  (2023-12-19T01:52:13Z)
• Supervised Metric Learning to Rank for Retrieval via Contextual Similarity Optimization [16.14184145802016]
  Many metric learning loss functions focus on learning a correct ranking of training samples, but strongly overfit semantically inconsistent labels. We propose a new metric learning method, called contextual loss, which optimize contextual similarity in addition to cosine similarity. We empirically show that the proposed loss is more robust to label noise, and is less prone to overfitting even when a large portion of train data is withheld.
  arXiv  Detail & Related papers  (2022-10-04T21:08:27Z)
• Benchmarking Deep AUROC Optimization: Loss Functions and Algorithmic Choices [37.559461866831754]
  We benchmark a variety of loss functions with different algorithmic choices for deep AUROC optimization problem. We highlight the essential choices such as positive sampling rate, regularization, normalization/activation, and weights. Our findings show that although Adam-type method is more competitive from training perspective, but it does not outperform others from testing perspective.
  arXiv  Detail & Related papers  (2022-03-27T00:47:00Z)
• Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance [53.49803579981569]
  We consider a global objective for contrastive learning, which contrasts each positive pair with all negative pairs for an anchor point. Existing methods such as SimCLR requires a large batch size in order to achieve a satisfactory result. We propose a memory-efficient optimization algorithm for solving the Global Contrastive Learning of Representations, named SogCLR.
  arXiv  Detail & Related papers  (2022-02-24T22:16:53Z)
• Adaptive Bandit Convex Optimization with Heterogeneous Curvature [40.368893108265056]
  We study a heterogeneous setting where each function has its own curvature that is only revealed after the learner makes a decision. We develop an efficient algorithm that is able to adapt to the curvature on the fly.
  arXiv  Detail & Related papers  (2022-02-12T21:55:42Z)
• Planning and Learning with Adaptive Lookahead [74.39132848733847]
  Policy Iteration (PI) algorithm alternates between greedy one-step policy improvement and policy evaluation. Recent literature shows that multi-step lookahead policy improvement leads to a better convergence rate at the expense of increased complexity per iteration. We propose for the first time to dynamically adapt the multi-step lookahead horizon as a function of the state and of the value estimate.
  arXiv  Detail & Related papers  (2022-01-28T20:26:55Z)
• Sample Complexity of Adversarially Robust Linear Classification on Separated Data [41.30425613353595]
  We consider the sample complexity of learning with adversarial robustness. We show that for very well-separated data, convergence rates of $O(frac1n)$ are achievable.
  arXiv  Detail & Related papers  (2020-12-19T22:04:59Z)
• Beyond Triplet Loss: Meta Prototypical N-tuple Loss for Person Re-identification [118.72423376789062]
  We introduce a multi-class classification loss, i.e., N-tuple loss, to jointly consider multiple (N) instances for per-query optimization. With the multi-class classification incorporated, our model achieves the state-of-the-art performance on the benchmark person ReID datasets.
  arXiv  Detail & Related papers  (2020-06-08T23:34:08Z)
• Ranking a set of objects: a graph based least-square approach [70.7866286425868]
  We consider the problem of ranking $N$ objects starting from a set of noisy pairwise comparisons provided by a crowd of equal workers. We propose a class of non-adaptive ranking algorithms that rely on a least-squares intrinsic optimization criterion for the estimation of qualities.
  arXiv  Detail & Related papers  (2020-02-26T16:19:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.