Multi-scale Knowledge Distillation for Unsupervised Person
Re-Identification
- URL: http://arxiv.org/abs/2204.09931v1
- Date: Thu, 21 Apr 2022 07:46:00 GMT
- Title: Multi-scale Knowledge Distillation for Unsupervised Person
Re-Identification
- Authors: Long Lan, Xiao Teng, Haoang Chi, Xiang Zhang
- Abstract summary: Unsupervised person re-identification is a challenging and promising task in the computer vision.
We take into account the features from two local views and produce multi-scale features.
Our approach achieves a state-of-the-art accuracy 85.7% @mAP or 94.3% @Rank-1 on the challenging Market-1501 benchmark with ResNet-50.
- Score: 9.70611259032691
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Unsupervised person re-identification is a challenging and promising task in
the computer vision. Nowadays unsupervised person re-identification methods
have achieved great improvements by training with pseudo labels. However, the
appearance and label noise are less explicitly studied in the unsupervised
manner. To relieve the effects of appearance noise the global features
involved, we also take into account the features from two local views and
produce multi-scale features. We explore the knowledge distillation to filter
label noise, Specifically, we first train a teacher model from noisy pseudo
labels in a iterative way, and then use the teacher model to guide the learning
of our student model. In our setting, the student model could converge fast in
the supervision of the teacher model thus reduce the interference of noisy
labels as the teacher model greatly suffered. After carefully handling the
noises in the feature learning, Our multi-scale knowledge distillation are
proven to be very effective in the unsupervised re-identification. Extensive
experiments on three popular person re-identification datasets demonstrate the
superiority of our method. Especially, our approach achieves a state-of-the-art
accuracy 85.7% @mAP or 94.3% @Rank-1 on the challenging Market-1501 benchmark
with ResNet-50 under the fully unsupervised setting.
Related papers
- Pseudo-label Correction for Instance-dependent Noise Using
Teacher-student Framework [1.2618527387900083]
We propose a new teacher-student based framework termed P-LC (pseudo-label correction)
In our novel approach, we reconfigure the teacher network into a triple encoder, leveraging the triplet loss to establish a pseudo-label correction system.
Experiments on MNIST, Fashion-MNIST, and SVHN demonstrate P-LC's superior performance over existing state-of-the-art methods across all noise levels.
arXiv Detail & Related papers (2023-11-24T00:36:17Z) - Exploring Inconsistent Knowledge Distillation for Object Detection with
Data Augmentation [66.25738680429463]
Knowledge Distillation (KD) for object detection aims to train a compact detector by transferring knowledge from a teacher model.
We propose inconsistent knowledge distillation (IKD) which aims to distill knowledge inherent in the teacher model's counter-intuitive perceptions.
Our method outperforms state-of-the-art KD baselines on one-stage, two-stage and anchor-free object detectors.
arXiv Detail & Related papers (2022-09-20T16:36:28Z) - Better Supervisory Signals by Observing Learning Paths [10.044413937134237]
We explain two existing label refining methods, label smoothing and knowledge distillation, in terms of our proposed criterion.
We observe the learning path, i.e., the trajectory of the model's predictions during training, for each training sample.
We find that the model can spontaneously refine "bad" labels through a "zig-zag" learning path, which occurs on both toy and real datasets.
arXiv Detail & Related papers (2022-03-04T18:31:23Z) - Anomaly Detection via Reverse Distillation from One-Class Embedding [2.715884199292287]
We propose a novel T-S model consisting of a teacher encoder and a student decoder.
Instead of receiving raw images directly, the student network takes teacher model's one-class embedding as input.
In addition, we introduce a trainable one-class bottleneck embedding module in our T-S model.
arXiv Detail & Related papers (2022-01-26T01:48:37Z) - Distantly-Supervised Named Entity Recognition with Noise-Robust Learning
and Language Model Augmented Self-Training [66.80558875393565]
We study the problem of training named entity recognition (NER) models using only distantly-labeled data.
We propose a noise-robust learning scheme comprised of a new loss function and a noisy label removal step.
Our method achieves superior performance, outperforming existing distantly-supervised NER models by significant margins.
arXiv Detail & Related papers (2021-09-10T17:19:56Z) - Unsupervised Noisy Tracklet Person Re-identification [100.85530419892333]
We present a novel selective tracklet learning (STL) approach that can train discriminative person re-id models from unlabelled tracklet data.
This avoids the tedious and costly process of exhaustively labelling person image/tracklet true matching pairs across camera views.
Our method is particularly more robust against arbitrary noisy data of raw tracklets therefore scalable to learning discriminative models from unconstrained tracking data.
arXiv Detail & Related papers (2021-01-16T07:31:00Z) - SEED: Self-supervised Distillation For Visual Representation [34.63488756535054]
We propose a new learning paradigm, named SElf-SupErvised Distillation (SEED), to transfer its representational knowledge into a smaller architecture (as Student) in a self-supervised fashion.
We show that SEED dramatically boosts the performance of small networks on downstream tasks.
arXiv Detail & Related papers (2021-01-12T20:04:50Z) - Distilling Object Detectors with Task Adaptive Regularization [97.52935611385179]
Current state-of-the-art object detectors are at the expense of high computational costs and are hard to deploy to low-end devices.
Knowledge distillation, which aims at training a smaller student network by transferring knowledge from a larger teacher model, is one of the promising solutions for model miniaturization.
arXiv Detail & Related papers (2020-06-23T15:58:22Z) - Knowledge Distillation Meets Self-Supervision [109.6400639148393]
Knowledge distillation involves extracting "dark knowledge" from a teacher network to guide the learning of a student network.
We show that the seemingly different self-supervision task can serve as a simple yet powerful solution.
By exploiting the similarity between those self-supervision signals as an auxiliary task, one can effectively transfer the hidden information from the teacher to the student.
arXiv Detail & Related papers (2020-06-12T12:18:52Z) - Learning Not to Learn in the Presence of Noisy Labels [104.7655376309784]
We show that a new class of loss functions called the gambler's loss provides strong robustness to label noise across various levels of corruption.
We show that training with this loss function encourages the model to "abstain" from learning on the data points with noisy labels.
arXiv Detail & Related papers (2020-02-16T09:12:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.