Rank-based loss for learning hierarchical representations
- URL: http://arxiv.org/abs/2110.05941v1
- Date: Mon, 11 Oct 2021 10:32:45 GMT
- Title: Rank-based loss for learning hierarchical representations
- Authors: Ines Nolasco and Dan Stowell
- Abstract summary: In machine learning, the family of methods that use the 'extra' information is called hierarchical classification.
Here we focus on how to integrate the hierarchical information of a problem to learn embeddings representative of the hierarchical relationships.
We show that rank based loss is suitable to learn hierarchical representations of the data.
- Score: 7.421724671710886
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Hierarchical taxonomies are common in many contexts, and they are a very
natural structure humans use to organise information. In machine learning, the
family of methods that use the 'extra' information is called hierarchical
classification. However, applied to audio classification, this remains
relatively unexplored. Here we focus on how to integrate the hierarchical
information of a problem to learn embeddings representative of the hierarchical
relationships. Previously, triplet loss has been proposed to address this
problem, however it presents some issues like requiring the careful
construction of the triplets, and being limited in the extent of hierarchical
information it uses at each iteration. In this work we propose a rank based
loss function that uses hierarchical information and translates this into a
rank ordering of target distances between the examples. We show that rank based
loss is suitable to learn hierarchical representations of the data. By testing
on unseen fine level classes we show that this method is also capable of
learning hierarchically correct representations of the new classes. Rank based
loss has two promising aspects, it is generalisable to hierarchies with any
number of levels, and is capable of dealing with data with incomplete
hierarchical labels.
Related papers
- Hierarchical Query Classification in E-commerce Search [38.67034103433015]
E-commerce platforms typically store and structure product information and search data in a hierarchy.
Efficiently categorizing user search queries into a similar hierarchical structure is paramount in enhancing user experience on e-commerce platforms as well as news curation and academic research.
The inherent complexity of hierarchical query classification is compounded by two primary challenges: (1) the pronounced class imbalance that skews towards dominant categories, and (2) the inherent brevity and ambiguity of search queries that hinder accurate classification.
arXiv Detail & Related papers (2024-03-09T21:55:55Z) - Learning List-Level Domain-Invariant Representations for Ranking [59.3544317373004]
We propose list-level alignment -- learning domain-invariant representations at the higher level of lists.
The benefits are twofold: it leads to the first domain adaptation generalization bound for ranking, in turn providing theoretical support for the proposed method.
arXiv Detail & Related papers (2022-12-21T04:49:55Z) - Use All The Labels: A Hierarchical Multi-Label Contrastive Learning
Framework [75.79736930414715]
We present a hierarchical multi-label representation learning framework that can leverage all available labels and preserve the hierarchical relationship between classes.
We introduce novel hierarchy preserving losses, which jointly apply a hierarchical penalty to the contrastive loss, and enforce the hierarchy constraint.
arXiv Detail & Related papers (2022-04-27T21:41:44Z) - Label Hierarchy Transition: Delving into Class Hierarchies to Enhance
Deep Classifiers [40.993137740456014]
We propose a unified probabilistic framework based on deep learning to address the challenges of hierarchical classification.
The proposed framework can be readily adapted to any existing deep network with only minor modifications.
We extend our proposed LHT framework to the skin lesion diagnosis task and validate its great potential in computer-aided diagnosis.
arXiv Detail & Related papers (2021-12-04T14:58:36Z) - TagRec: Automated Tagging of Questions with Hierarchical Learning
Taxonomy [0.0]
Online educational platforms organize academic questions based on a hierarchical learning taxonomy (subject-chapter-topic)
This paper formulates the problem as a similarity-based retrieval task where we optimize the semantic relatedness between the taxonomy and the questions.
We demonstrate that our method helps to handle the unseen labels and hence can be used for taxonomy tagging in the wild.
arXiv Detail & Related papers (2021-07-03T11:50:55Z) - MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
MATCH is an end-to-end framework that leverages both metadata and hierarchy information.
We propose different ways to regularize the parameters and output probability of each child label by its parents.
Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
arXiv Detail & Related papers (2021-02-15T05:23:08Z) - Pitfalls of Assessing Extracted Hierarchies for Multi-Class
Classification [4.89253144446913]
We identify some common pitfalls that may lead practitioners to make misleading conclusions about their methods.
We show how the hierarchy's quality can become irrelevant depending on the experimental setup.
Our results confirm that datasets with a high number of classes generally present complex structures in how these classes relate to each other.
arXiv Detail & Related papers (2021-01-26T21:50:57Z) - Noisy Labels Can Induce Good Representations [53.47668632785373]
We study how architecture affects learning with noisy labels.
We show that training with noisy labels can induce useful hidden representations, even when the model generalizes poorly.
This finding leads to a simple method to improve models trained on noisy labels.
arXiv Detail & Related papers (2020-12-23T18:58:05Z) - Exploring the Hierarchy in Relation Labels for Scene Graph Generation [75.88758055269948]
The proposed method can improve several state-of-the-art baselines by a large margin (up to $33%$ relative gain) in terms of Recall@50.
Experiments show that the proposed simple yet effective method can improve several state-of-the-art baselines by a large margin.
arXiv Detail & Related papers (2020-09-12T17:36:53Z) - Hierarchical Class-Based Curriculum Loss [18.941207332233805]
Most real world data have dependencies between labels, which can be captured by using a hierarchy.
We propose a loss function, hierarchical curriculum loss, with two properties: (i) satisfy hierarchical constraints present in the label space, and (ii) provide non-uniform weights to labels based on their levels in the hierarchy.
arXiv Detail & Related papers (2020-06-05T18:48:57Z) - Structured Prediction with Partial Labelling through the Infimum Loss [85.4940853372503]
The goal of weak supervision is to enable models to learn using only forms of labelling which are cheaper to collect.
This is a type of incomplete annotation where, for each datapoint, supervision is cast as a set of labels containing the real one.
This paper provides a unified framework based on structured prediction and on the concept of infimum loss to deal with partial labelling.
arXiv Detail & Related papers (2020-03-02T13:59:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.