Revisiting lp-constrained Softmax Loss: A Comprehensive Study

• URL: http://arxiv.org/abs/2206.09616v1
• Date: Mon, 20 Jun 2022 08:03:12 GMT
• Title: Revisiting lp-constrained Softmax Loss: A Comprehensive Study
• Authors: Chintan Trivedi, Konstantinos Makantasis, Antonios Liapis, Georgios N. Yannakakis
• Abstract summary: We investigate the performance of lp-constrained softmax loss classifiers across different norm orders, magnitudes, and data dimensions. Experimental results suggest collectively that lp-constrained softmax loss classifiers can achieve more accurate classification results. We suggest that lp normalization is a recommended data representation practice for image classification in terms of performance and convergence.
• Score: 2.570570340104555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Normalization is a vital process for any machine learning task as it controls the properties of data and affects model performance at large. The impact of particular forms of normalization, however, has so far been investigated in limited domain-specific classification tasks and not in a general fashion. Motivated by the lack of such a comprehensive study, in this paper we investigate the performance of lp-constrained softmax loss classifiers across different norm orders, magnitudes, and data dimensions in both proof-of-concept classification problems and real-world popular image classification tasks. Experimental results suggest collectively that lp-constrained softmax loss classifiers not only can achieve more accurate classification results but, at the same time, appear to be less prone to overfitting. The core findings hold across the three popular deep learning architectures tested and eight datasets examined, and suggest that lp normalization is a recommended data representation practice for image classification in terms of performance and convergence, and against overfitting.

Related papers

• Is Softmax Loss All You Need? A Principled Analysis of Softmax-family Loss [91.61796429377041]
  The Softmax loss is one of the most widely employed surrogate objectives for classification and ranking tasks.<n>We investigate whether different surrogates achieve consistency with classification and ranking metrics, and analyze their gradient dynamics to reveal distinct convergence behaviors.<n>Our results establish a principled foundation and offer practical guidance for loss selections in large-class machine learning applications.
  arXiv  Detail & Related papers  (2026-01-30T09:24:52Z)
• Removing Geometric Bias in One-Class Anomaly Detection with Adaptive Feature Perturbation [10.556297392824721]
  One-class anomaly detection aims to detect objects that do not belong to a predefined normal class.<n>We consider frozen yet rich feature spaces given by pretrained models and create pseudo-anomalous features with a novel adaptive linear feature perturbation technique.<n>It adapts the noise distribution to each sample applies decaying linear perturbations to feature vectors and further guides the classification process using a contrastive learning objective.
  arXiv  Detail & Related papers  (2025-03-07T15:42:51Z)
• Deep Subspace Learning for Surface Anomaly Classification Based on 3D Point Cloud Data [2.5524809198548137]
  This paper proposes a novel deep subspace learning-based 3D anomaly classification model. Specifically, we model each class as a subspace to account for the intra-class variation, while promoting distinct subspaces of different classes to tackle the inter-class similarity. Our method achieves better anomaly classification results than benchmark methods, and can effectively identify the new types of anomalies.
  arXiv  Detail & Related papers  (2025-02-17T10:57:53Z)
• Breaking Fine-Grained Classification Barriers with Cost-Free Data in Few-Shot Class-Incremental Learning [13.805180905579832]
  We propose a novel learning paradigm to break barriers in fine-grained classification. It enables the model to learn beyond the standard training phase and benefit from cost-free data encountered during system operation.
  arXiv  Detail & Related papers  (2024-12-29T07:11:44Z)
• Fine-grained Abnormality Prompt Learning for Zero-shot Anomaly Detection [88.34095233600719]
  FAPrompt is a novel framework designed to learn Fine-grained Abnormality Prompts for more accurate ZSAD. It substantially outperforms state-of-the-art methods by at least 3%-5% AUC/AP in both image- and pixel-level ZSAD tasks.
  arXiv  Detail & Related papers  (2024-10-14T08:41:31Z)
• Fine-Grained ImageNet Classification in the Wild [0.0]
  Robustness tests can uncover several vulnerabilities and biases which go unnoticed during the typical model evaluation stage. In our work, we perform fine-grained classification on closely related categories, which are identified with the help of hierarchical knowledge.
  arXiv  Detail & Related papers  (2023-03-04T12:25:07Z)
• Fine-grained Retrieval Prompt Tuning [149.9071858259279]
  Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation. Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
  arXiv  Detail & Related papers  (2022-07-29T04:10:04Z)
• Prototype-Anchored Learning for Learning with Imperfect Annotations [83.7763875464011]
  It is challenging to learn unbiased classification models from imperfectly annotated datasets. We propose a prototype-anchored learning (PAL) method, which can be easily incorporated into various learning-based classification schemes. We verify the effectiveness of PAL on class-imbalanced learning and noise-tolerant learning by extensive experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-06-23T10:25:37Z)
• Few-shot Deep Representation Learning based on Information Bottleneck Principle [0.0]
  In a standard anomaly detection problem, a detection model is trained in an unsupervised setting, under an assumption that the samples were generated from a single source of normal data. In practice, normal data often consist of multiple classes. In such settings, learning to differentiate between normal instances and anomalies among discrepancies between normal classes without large-scale labeled data presents a significant challenge. In this work, we attempt to overcome this challenge by preparing few examples from each normal class, which is not excessively costly.
  arXiv  Detail & Related papers  (2021-11-25T07:15:12Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• Enhancing Fine-Grained Classification for Low Resolution Images [97.82441158440527]
  Low resolution images suffer from the inherent challenge of limited information content and the absence of fine details useful for sub-category classification. This research proposes a novel attribute-assisted loss, which utilizes ancillary information to learn discriminative features for classification. The proposed loss function enables a model to learn class-specific discriminative features, while incorporating attribute-level separability.
  arXiv  Detail & Related papers  (2021-05-01T13:19:02Z)
• von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning [12.37528281037283]
  Recent work has argued that classification losses utilizing softmax cross-entropy are superior not only for fixed-set classification tasks, but also for open-set tasks. We conduct an empirical investigation of embedding geometry on softmax losses for a variety of fixed-set classification and image retrieval tasks.
  arXiv  Detail & Related papers  (2021-03-29T16:07:09Z)
• Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View [82.80085730891126]
  We provide the first modernally precise analysis of linear multiclass classification. Our analysis reveals that the classification accuracy is highly distribution-dependent. The insights gained may pave the way for a precise understanding of other classification algorithms.
  arXiv  Detail & Related papers  (2020-11-16T05:17:29Z)
• Contraction Mapping of Feature Norms for Classifier Learning on the Data with Different Quality [5.47982638565422]
  We propose a contraction mapping function to compress the range of feature norms of training images according to their quality. Experiments on various classification applications, including handwritten digit recognition, lung nodule classification, face verification and face recognition, demonstrate that the proposed approach is promising.
  arXiv  Detail & Related papers  (2020-07-27T09:53:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.