Propheter: Prophetic Teacher Guided Long-Tailed Distribution Learning
- URL: http://arxiv.org/abs/2304.04135v2
- Date: Mon, 25 Sep 2023 08:28:27 GMT
- Title: Propheter: Prophetic Teacher Guided Long-Tailed Distribution Learning
- Authors: Wenxiang Xu, Yongcheng Jing, Linyun Zhou, Wenqi Huang, Lechao Cheng,
Zunlei Feng, Mingli Song
- Abstract summary: We propose an innovative long-tailed learning paradigm that breaks the bottleneck by guiding the learning of deep networks with external prior knowledge.
The proposed prophetic paradigm acts as a promising solution to the challenge of limited class knowledge in long-tailed datasets.
- Score: 44.947984354108094
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The problem of deep long-tailed learning, a prevalent challenge in the realm
of generic visual recognition, persists in a multitude of real-world
applications. To tackle the heavily-skewed dataset issue in long-tailed
classification, prior efforts have sought to augment existing deep models with
the elaborate class-balancing strategies, such as class rebalancing, data
augmentation, and module improvement. Despite the encouraging performance, the
limited class knowledge of the tailed classes in the training dataset still
bottlenecks the performance of the existing deep models. In this paper, we
propose an innovative long-tailed learning paradigm that breaks the bottleneck
by guiding the learning of deep networks with external prior knowledge. This is
specifically achieved by devising an elaborated ``prophetic'' teacher, termed
as ``Propheter'', that aims to learn the potential class distributions. The
target long-tailed prediction model is then optimized under the instruction of
the well-trained ``Propheter'', such that the distributions of different
classes are as distinguishable as possible from each other. Experiments on
eight long-tailed benchmarks across three architectures demonstrate that the
proposed prophetic paradigm acts as a promising solution to the challenge of
limited class knowledge in long-tailed datasets. The developed code is publicly
available at \url{https://github.com/tcmyxc/propheter}.
Related papers
- Granularity Matters in Long-Tail Learning [62.30734737735273]
We offer a novel perspective on long-tail learning, inspired by an observation: datasets with finer granularity tend to be less affected by data imbalance.
We introduce open-set auxiliary classes that are visually similar to existing ones, aiming to enhance representation learning for both head and tail classes.
To prevent the overwhelming presence of auxiliary classes from disrupting training, we introduce a neighbor-silencing loss.
arXiv Detail & Related papers (2024-10-21T13:06:21Z) - TaE: Task-aware Expandable Representation for Long Tail Class Incremental Learning [42.630413950957795]
We introduce a novel Task-aware Expandable (TaE) framework to learn diverse representations from each incremental task.
TaE achieves state-of-the-art performance.
arXiv Detail & Related papers (2024-02-08T16:37:04Z) - Robust Feature Learning and Global Variance-Driven Classifier Alignment
for Long-Tail Class Incremental Learning [20.267257778779992]
This paper introduces a two-stage framework designed to enhance long-tail class incremental learning.
We address the challenge posed by the under-representation of tail classes in long-tail class incremental learning.
The proposed framework can seamlessly integrate as a module with any class incremental learning method.
arXiv Detail & Related papers (2023-11-02T13:28:53Z) - Learning to Jump: Thinning and Thickening Latent Counts for Generative
Modeling [69.60713300418467]
Learning to jump is a general recipe for generative modeling of various types of data.
We demonstrate when learning to jump is expected to perform comparably to learning to denoise, and when it is expected to perform better.
arXiv Detail & Related papers (2023-05-28T05:38:28Z) - Deep networks for system identification: a Survey [56.34005280792013]
System identification learns mathematical descriptions of dynamic systems from input-output data.
Main aim of the identified model is to predict new data from previous observations.
We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
arXiv Detail & Related papers (2023-01-30T12:38:31Z) - Plausible Counterfactuals: Auditing Deep Learning Classifiers with
Realistic Adversarial Examples [84.8370546614042]
Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data.
Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model.
Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
arXiv Detail & Related papers (2020-03-25T11:08:56Z) - Learning From Multiple Experts: Self-paced Knowledge Distillation for
Long-tailed Classification [106.08067870620218]
We propose a self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME)
We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model.
We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods.
arXiv Detail & Related papers (2020-01-06T12:57:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.