Hybrid Losses for Hierarchical Embedding Learning

• URL: http://arxiv.org/abs/2501.12796v1
• Date: Wed, 22 Jan 2025 10:58:04 GMT
• Title: Hybrid Losses for Hierarchical Embedding Learning
• Authors: Haokun Tian, Stefan Lattner, Brian McFee, Charalampos Saitis,
• Abstract summary: We investigate hybrid losses, such as generalised triplet and cross-entropy losses, within a multi-task learning framework. Our experiments on OrchideaSOL, a four-level hierarchical instrument sound dataset, demonstrate that the proposed hybrid losses outperform previous works in classification, retrieval, embedding space structure, and generalisation.
• Score: 4.2525210928495625
• License:
• Abstract: In traditional supervised learning, the cross-entropy loss treats all incorrect predictions equally, ignoring the relevance or proximity of wrong labels to the correct answer. By leveraging a tree hierarchy for fine-grained labels, we investigate hybrid losses, such as generalised triplet and cross-entropy losses, to enforce similarity between labels within a multi-task learning framework. We propose metrics to evaluate the embedding space structure and assess the model's ability to generalise to unseen classes, that is, to infer similar classes for data belonging to unseen categories. Our experiments on OrchideaSOL, a four-level hierarchical instrument sound dataset with nearly 200 detailed categories, demonstrate that the proposed hybrid losses outperform previous works in classification, retrieval, embedding space structure, and generalisation.

Related papers

• Bidirectional Logits Tree: Pursuing Granularity Reconcilement in Fine-Grained Classification [89.20477310885731]
  This paper addresses the challenge of Granularity Competition in fine-grained classification tasks. Existing approaches typically develop independent hierarchy-aware models based on shared features extracted from a common base encoder. We propose a novel framework called the Bidirectional Logits Tree (BiLT) for Granularity Reconcilement.
  arXiv  Detail & Related papers  (2024-12-17T10:42:19Z)
• Harnessing Superclasses for Learning from Hierarchical Databases [1.835004446596942]
  In many large-scale classification problems, classes are organized in a known hierarchy, typically represented as a tree. We introduce a loss for this type of supervised hierarchical classification. Our approach does not entail any significant additional computational cost compared with the loss of cross-entropy.
  arXiv  Detail & Related papers  (2024-11-25T14:39:52Z)
• Hierarchical confusion matrix for classification performance evaluation [0.0]
  We develop the concept of a hierarchical confusion matrix and prove its applicability to all types of hierarchical classification problems. We use measures based on the novel confusion matrix to evaluate models within a benchmark for three real world hierarchical classification applications. The results outline the reasonability of this approach and its usefulness to evaluate hierarchical classification problems.
  arXiv  Detail & Related papers  (2023-06-15T19:31:59Z)
• Weakly-supervised Action Localization via Hierarchical Mining [76.00021423700497]
  Weakly-supervised action localization aims to localize and classify action instances in the given videos temporally with only video-level categorical labels. We propose a hierarchical mining strategy under video-level and snippet-level manners, i.e., hierarchical supervision and hierarchical consistency mining. We show that HiM-Net outperforms existing methods on THUMOS14 and ActivityNet1.3 datasets with large margins by hierarchically mining the supervision and consistency.
  arXiv  Detail & Related papers  (2022-06-22T12:19:09Z)
• Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework [75.79736930414715]
  We present a hierarchical multi-label representation learning framework that can leverage all available labels and preserve the hierarchical relationship between classes. We introduce novel hierarchy preserving losses, which jointly apply a hierarchical penalty to the contrastive loss, and enforce the hierarchy constraint.
  arXiv  Detail & Related papers  (2022-04-27T21:41:44Z)
• Resolving label uncertainty with implicit posterior models [71.62113762278963]
  We propose a method for jointly inferring labels across a collection of data samples. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs.
  arXiv  Detail & Related papers  (2022-02-28T18:09:44Z)
• Inducing a hierarchy for multi-class classification problems [11.58041597483471]
  In applications where categorical labels follow a natural hierarchy, classification methods that exploit the label structure often outperform those that do not. In this paper, we investigate a class of methods that induce a hierarchy that can similarly improve classification performance over flat classifiers. We demonstrate the effectiveness of the class of methods both for discovering a latent hierarchy and for improving accuracy in principled simulation settings and three real data applications.
  arXiv  Detail & Related papers  (2021-02-20T05:40:42Z)
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function [106.69643619725652]
  We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results. We report state-of-the-art results for text classification task on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-08T21:55:22Z)
• Rethinking preventing class-collapsing in metric learning with margin-based losses [81.22825616879936]
  Metric learning seeks embeddings where visually similar instances are close and dissimilar instances are apart. margin-based losses tend to project all samples of a class onto a single point in the embedding space. We propose a simple modification to the embedding losses such that each sample selects its nearest same-class counterpart in a batch.
  arXiv  Detail & Related papers  (2020-06-09T09:59:25Z)
• Hierarchical Class-Based Curriculum Loss [18.941207332233805]
  Most real world data have dependencies between labels, which can be captured by using a hierarchy. We propose a loss function, hierarchical curriculum loss, with two properties: (i) satisfy hierarchical constraints present in the label space, and (ii) provide non-uniform weights to labels based on their levels in the hierarchy.
  arXiv  Detail & Related papers  (2020-06-05T18:48:57Z)
• Hierarchical Entity Typing via Multi-level Learning to Rank [38.509244927293715]
  We propose a novel method for hierarchical entity classification that embraces ontological structure at both training and during prediction. At training, our novel multi-level learning-to-rank loss compares positive types against negative siblings according to the type tree. During prediction, we define a coarse-to-fine decoder that restricts viable candidates at each level of the ontology based on already predicted parent type(s)
  arXiv  Detail & Related papers  (2020-04-05T19:27:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.