Orthogonal Projection Loss

• URL: http://arxiv.org/abs/2103.14021v1
• Date: Thu, 25 Mar 2021 17:58:00 GMT
• Title: Orthogonal Projection Loss
• Authors: Kanchana Ranasinghe, Muzammal Naseer, Munawar Hayat, Salman Khan, Fahad Shahbaz Khan
• Abstract summary: We develop a novel loss function termed Orthogonal Projection Loss' (OPL) OPL directly enforces inter-class separation alongside intra-class clustering in the feature space. OPL offers unique advantages as it does not require careful negative mining and is not sensitive to the batch size.
• Score: 59.61277381836491
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks have achieved remarkable performance on a range of classification tasks, with softmax cross-entropy (CE) loss emerging as the de-facto objective function. The CE loss encourages features of a class to have a higher projection score on the true class-vector compared to the negative classes. However, this is a relative constraint and does not explicitly force different class features to be well-separated. Motivated by the observation that ground-truth class representations in CE loss are orthogonal (one-hot encoded vectors), we develop a novel loss function termed `Orthogonal Projection Loss' (OPL) which imposes orthogonality in the feature space. OPL augments the properties of CE loss and directly enforces inter-class separation alongside intra-class clustering in the feature space through orthogonality constraints on the mini-batch level. As compared to other alternatives of CE, OPL offers unique advantages e.g., no additional learnable parameters, does not require careful negative mining and is not sensitive to the batch size. Given the plug-and-play nature of OPL, we evaluate it on a diverse range of tasks including image recognition (CIFAR-100), large-scale classification (ImageNet), domain generalization (PACS) and few-shot learning (miniImageNet, CIFAR-FS, tiered-ImageNet and Meta-dataset) and demonstrate its effectiveness across the board. Furthermore, OPL offers better robustness against practical nuisances such as adversarial attacks and label noise. Code is available at: https://github.com/kahnchana/opl.

Related papers

• Zero-shot Object Counting with Good Exemplars [35.7544908318547]
  Zero-shot object counting (ZOC) aims to enumerate objects in images using only the names of object classes during testing, without the need for manual annotations. We propose the Visual Association-based Zero-shot Object Counting (VA-Count) framework. VA-Count consists of an Exemplar Enhancement Module (EEM) and a Noise Suppression Module (NSM) that synergistically refine the process of class exemplar identification while minimizing the consequences of incorrect object identification.
  arXiv  Detail & Related papers  (2024-07-06T03:37:22Z)
• Exploration of Class Center for Fine-Grained Visual Classification [7.120809788357261]
  Fine-grained visual classification is a challenging task due to intra-class variances and subtle inter-class differences. We propose a loss function named exploration of class center, which consists of a multiple class-center constraint and a class-center label generation. Our method can be easily integrated with existing fine-grained visual classification approaches as a loss function, to further boost excellent performance with only slight training costs.
  arXiv  Detail & Related papers  (2024-07-05T04:11:09Z)
• Class Anchor Margin Loss for Content-Based Image Retrieval [97.81742911657497]
  We propose a novel repeller-attractor loss that falls in the metric learning paradigm, yet directly optimize for the L2 metric without the need of generating pairs. We evaluate the proposed objective in the context of few-shot and full-set training on the CBIR task, by using both convolutional and transformer architectures.
  arXiv  Detail & Related papers  (2023-06-01T12:53:10Z)
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning [120.53458753007851]
  Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. We deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse. We propose a neural collapse inspired framework for FSCIL. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances.
  arXiv  Detail & Related papers  (2023-02-06T18:39:40Z)
• Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning [23.465747123791772]
  In few-shot classification, the primary goal is to learn representations that generalize well for novel classes. We propose an efficient low displacement rank (LDR) regularization strategy termed Ortho-Shot.
  arXiv  Detail & Related papers  (2021-10-18T14:58:36Z)
• Discriminator-Free Generative Adversarial Attack [87.71852388383242]
  Agenerative-based adversarial attacks can get rid of this limitation. ASymmetric Saliency-based Auto-Encoder (SSAE) generates the perturbations. The adversarial examples generated by SSAE not only make thewidely-used models collapse, but also achieves good visual quality.
  arXiv  Detail & Related papers  (2021-07-20T01:55:21Z)
• Learning Class Unique Features in Fine-Grained Visual Classification [20.59233720331779]
  We propose to regularize the training of CNN by enforcing the uniqueness of the features to each category from an information theoretic perspective. We present a Feature Redundancy Loss (FRL) based on normalized inner product between each selected feature map pair to complement the proposed minimax loss.
  arXiv  Detail & Related papers  (2020-11-22T06:20:47Z)
• Class Interference Regularization [7.248447600071719]
  Contrastive losses yield state-of-the-art performance for person re-identification, face verification and few shot learning. We propose a novel, simple and effective regularization technique, the Class Interference Regularization (CIR) CIR perturbs the output features by randomly moving them towards the average embeddings of the negative classes.
  arXiv  Detail & Related papers  (2020-09-04T21:03:32Z)
• Generalized Zero-Shot Learning Via Over-Complete Distribution [79.5140590952889]
  We propose to generate an Over-Complete Distribution (OCD) using Conditional Variational Autoencoder (CVAE) of both seen and unseen classes. The effectiveness of the framework is evaluated using both Zero-Shot Learning and Generalized Zero-Shot Learning protocols.
  arXiv  Detail & Related papers  (2020-04-01T19:05:28Z)
• Improving Few-shot Learning by Spatially-aware Matching and CrossTransformer [116.46533207849619]
  We study the impact of scale and location mismatch in the few-shot learning scenario. We propose a novel Spatially-aware Matching scheme to effectively perform matching across multiple scales and locations.
  arXiv  Detail & Related papers  (2020-01-06T14:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.