SuSana Distancia is all you need: Enforcing class separability in metric
learning via two novel distance-based loss functions for few-shot image
classification
- URL: http://arxiv.org/abs/2305.09062v3
- Date: Thu, 18 May 2023 20:41:34 GMT
- Title: SuSana Distancia is all you need: Enforcing class separability in metric
learning via two novel distance-based loss functions for few-shot image
classification
- Authors: Mauricio Mendez-Ruiz, Jorge Gonzalez-Zapata, Ivan Reyes-Amezcua,
Daniel Flores-Araiza, Francisco Lopez-Tiro, Andres Mendez-Vazquez, Gilberto
Ochoa-Ruiz
- Abstract summary: We propose two loss functions which consider the importance of the embedding vectors by looking at the intra-class and inter-class distance between the few data.
Our results show a significant improvement in accuracy in the miniImagenNet benchmark compared to other metric-based few-shot learning methods by a margin of 2%.
- Score: 0.9236074230806579
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Few-shot learning is a challenging area of research that aims to learn new
concepts with only a few labeled samples of data. Recent works based on
metric-learning approaches leverage the meta-learning approach, which is
encompassed by episodic tasks that make use a support (training) and query set
(test) with the objective of learning a similarity comparison metric between
those sets. Due to the lack of data, the learning process of the embedding
network becomes an important part of the few-shot task. Previous works have
addressed this problem using metric learning approaches, but the properties of
the underlying latent space and the separability of the difference classes on
it was not entirely enforced. In this work, we propose two different loss
functions which consider the importance of the embedding vectors by looking at
the intra-class and inter-class distance between the few data. The first loss
function is the Proto-Triplet Loss, which is based on the original triplet loss
with the modifications needed to better work on few-shot scenarios. The second
loss function, which we dub ICNN loss is based on an inter and intra class
nearest neighbors score, which help us to assess the quality of embeddings
obtained from the trained network. Our results, obtained from a extensive
experimental setup show a significant improvement in accuracy in the
miniImagenNet benchmark compared to other metric-based few-shot learning
methods by a margin of 2%, demonstrating the capability of these loss functions
to allow the network to generalize better to previously unseen classes. In our
experiments, we demonstrate competitive generalization capabilities to other
domains, such as the Caltech CUB, Dogs and Cars datasets compared with the
state of the art.
Related papers
- Class Anchor Margin Loss for Content-Based Image Retrieval [97.81742911657497]
We propose a novel repeller-attractor loss that falls in the metric learning paradigm, yet directly optimize for the L2 metric without the need of generating pairs.
We evaluate the proposed objective in the context of few-shot and full-set training on the CBIR task, by using both convolutional and transformer architectures.
arXiv Detail & Related papers (2023-06-01T12:53:10Z) - Adaptive Cross Batch Normalization for Metric Learning [75.91093210956116]
Metric learning is a fundamental problem in computer vision.
We show that it is equally important to ensure that the accumulated embeddings are up to date.
In particular, it is necessary to circumvent the representational drift between the accumulated embeddings and the feature embeddings at the current training iteration.
arXiv Detail & Related papers (2023-03-30T03:22:52Z) - Relational Surrogate Loss Learning [41.61184221367546]
This paper revisits the surrogate loss learning, where a deep neural network is employed to approximate the evaluation metrics.
In this paper, we show that directly maintaining the relation of models between surrogate losses and metrics suffices.
Our method is much easier to optimize and enjoys significant efficiency and performance gains.
arXiv Detail & Related papers (2022-02-26T17:32:57Z) - Incremental Class Learning using Variational Autoencoders with
Similarity Learning [0.0]
Catastrophic forgetting in neural networks during incremental learning remains a challenging problem.
Our research investigates catastrophic forgetting for four well-known metric-based loss functions during incremental class learning.
The angular loss was least affected, followed by contrastive, triplet loss, and centre loss with good mining techniques.
arXiv Detail & Related papers (2021-10-04T10:19:53Z) - Finding Significant Features for Few-Shot Learning using Dimensionality
Reduction [0.0]
This module helps to improve the accuracy performance by allowing the similarity function, given by the metric learning method, to have more discriminative features for the classification.
Our method outperforms the metric learning baselines in the miniImageNet dataset by around 2% in accuracy performance.
arXiv Detail & Related papers (2021-07-06T16:36:57Z) - Few-shot Action Recognition with Prototype-centered Attentive Learning [88.10852114988829]
Prototype-centered Attentive Learning (PAL) model composed of two novel components.
First, a prototype-centered contrastive learning loss is introduced to complement the conventional query-centered learning objective.
Second, PAL integrates a attentive hybrid learning mechanism that can minimize the negative impacts of outliers.
arXiv Detail & Related papers (2021-01-20T11:48:12Z) - Why Do Better Loss Functions Lead to Less Transferable Features? [93.47297944685114]
This paper studies how the choice of training objective affects the transferability of the hidden representations of convolutional neural networks trained on ImageNet.
We show that many objectives lead to statistically significant improvements in ImageNet accuracy over vanilla softmax cross-entropy, but the resulting fixed feature extractors transfer substantially worse to downstream tasks.
arXiv Detail & Related papers (2020-10-30T17:50:31Z) - ReMarNet: Conjoint Relation and Margin Learning for Small-Sample Image
Classification [49.87503122462432]
We introduce a novel neural network termed Relation-and-Margin learning Network (ReMarNet)
Our method assembles two networks of different backbones so as to learn the features that can perform excellently in both of the aforementioned two classification mechanisms.
Experiments on four image datasets demonstrate that our approach is effective in learning discriminative features from a small set of labeled samples.
arXiv Detail & Related papers (2020-06-27T13:50:20Z) - Learning to hash with semantic similarity metrics and empirical KL
divergence [3.04585143845864]
Learning to hash is an efficient paradigm for exact and approximate nearest neighbor search from massive databases.
Binary hash codes are typically extracted from an image by rounding output features from a CNN, which is trained on a supervised binary similar/ dissimilar task.
We overcome (i) via a novel loss function encouraging the relative hash code distances of learned features to match those derived from their targets.
We address (ii) via a differentiable estimate of the KL divergence between network outputs and a binary target distribution, resulting in minimal information loss when the features are rounded to binary.
arXiv Detail & Related papers (2020-05-11T08:20:26Z) - One-Shot Object Detection without Fine-Tuning [62.39210447209698]
We introduce a two-stage model consisting of a first stage Matching-FCOS network and a second stage Structure-Aware Relation Module.
We also propose novel training strategies that effectively improve detection performance.
Our method exceeds the state-of-the-art one-shot performance consistently on multiple datasets.
arXiv Detail & Related papers (2020-05-08T01:59:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.