Real Additive Margin Softmax for Speaker Verification

• URL: http://arxiv.org/abs/2110.09116v1
• Date: Mon, 18 Oct 2021 09:11:14 GMT
• Title: Real Additive Margin Softmax for Speaker Verification
• Authors: Lantian Li and Ruiqian Nai and Dong Wang
• Abstract summary: We show that AM-Softmax loss does not implement real max-margin training. We present a Real AM-Softmax loss which involves a true margin function in the softmax training.
• Score: 14.226089039985151
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The additive margin softmax (AM-Softmax) loss has delivered remarkable performance in speaker verification. A supposed behavior of AM-Softmax is that it can shrink within-class variation by putting emphasis on target logits, which in turn improves margin between target and non-target classes. In this paper, we conduct a careful analysis on the behavior of AM-Softmax loss, and show that this loss does not implement real max-margin training. Based on this observation, we present a Real AM-Softmax loss which involves a true margin function in the softmax training. Experiments conducted on VoxCeleb1, SITW and CNCeleb demonstrated that the corrected AM-Softmax loss consistently outperforms the original one. The code has been released at https://gitlab.com/csltstu/sunine.

Related papers

• MultiMax: Sparse and Multi-Modal Attention Learning [60.49318008131978]
  SoftMax is a ubiquitous ingredient of modern machine learning algorithms. We show that sparsity can be achieved by a family of SoftMax variants, but they often require an alternative loss function and do not preserve multi-modality. We propose MultiMax, which adaptively modulates the output distribution according to input entry range.
  arXiv  Detail & Related papers  (2024-06-03T10:51:43Z)
• Spectral Aware Softmax for Visible-Infrared Person Re-Identification [123.69049942659285]
  Visible-infrared person re-identification (VI-ReID) aims to match specific pedestrian images from different modalities. Existing methods still follow the softmax loss training paradigm, which is widely used in single-modality classification tasks. We propose the spectral-aware softmax (SA-Softmax) loss, which can fully explore the embedding space with the modality information.
  arXiv  Detail & Related papers  (2023-02-03T02:57:18Z)
• NBC-Softmax : Darkweb Author fingerprinting and migration tracking [1.1470070927586016]
  Metric learning aims to learn distances from the data, which enhances the performance of similarity-based algorithms. We propose NBC-Softmax, a contrastive loss based clustering technique for softmax loss. Our technique meets the criterion for larger number of samples, thus achieving block contrastiveness.
  arXiv  Detail & Related papers  (2022-12-15T23:00:33Z)
• Understanding Softmax Confidence and Uncertainty [95.71801498763216]
  It is often remarked that neural networks fail to increase their uncertainty when predicting on data far from the training distribution. Yet naively using softmax confidence as a proxy for uncertainty achieves modest success in tasks exclusively testing for this. This paper investigates this contradiction, identifying two implicit biases that do encourage softmax confidence to correlate with uncertainty.
  arXiv  Detail & Related papers  (2021-06-09T10:37:29Z)
• Max-Margin is Dead, Long Live Max-Margin! [87.90853526750716]
  We show that Max-Margin loss can only be consistent to the classification task under highly restrictive assumptions on the discrete loss measuring the error between distances. We introduce Restricted-Max-Margin, where the loss-augmented scores is maintained, but performed over a subset of the original domain.
  arXiv  Detail & Related papers  (2021-05-31T15:55:52Z)
• Exploring Alternatives to Softmax Function [0.5924831288313849]
  We investigate Taylor softmax, SM-softmax and our proposed SM-Taylor softmax as alternatives to softmax function. Our experiments for the image classification task on different datasets reveal that there is always a configuration of the SM-Taylor softmax function that outperforms the normal softmax function.
  arXiv  Detail & Related papers  (2020-11-23T16:50:18Z)
• Imbalance Robust Softmax for Deep Embeeding Learning [34.95520933299555]
  In recent years, one research focus is to solve the open-set problem by discriminative deep embedding learning in the field of face recognition (FR) and person re-identification (re-ID) We find that imbalanced training data is another main factor causing the performance of FR and re-ID with softmax or its variants. We propose a unified framework, Imbalance-Robust Softmax (IR-Softmax), which can simultaneously solve the open-set problem and reduce the influence of data imbalance.
  arXiv  Detail & Related papers  (2020-11-23T00:43:07Z)
• Balanced Meta-Softmax for Long-Tailed Visual Recognition [46.215759445665434]
  We show that the Softmax function, though used in most classification tasks, gives a biased gradient estimation under the long-tailed setup. This paper presents Balanced Softmax, an elegant unbiased extension of Softmax, to accommodate the label distribution shift between training and testing. In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.
  arXiv  Detail & Related papers  (2020-07-21T12:05:00Z)
• Taming GANs with Lookahead-Minmax [63.90038365274479]
  Experimental results on MNIST, SVHN, CIFAR-10, and ImageNet demonstrate a clear advantage of combining Lookahead-minmax with Adam or extragradient. Using 30-fold fewer parameters and 16-fold smaller minibatches we outperform the reported performance of the class-dependent BigGAN on CIFAR-10 by obtaining FID of 12.19 without using the class labels.
  arXiv  Detail & Related papers  (2020-06-25T17:13:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.