Related papers: Multi-Objective Interpolation Training for Robustness to Label Noise

Multi-Objective Interpolation Training for Robustness to Label Noise

URL: http://arxiv.org/abs/2012.04462v2
Date: Thu, 18 Mar 2021 07:44:28 GMT
Title: Multi-Objective Interpolation Training for Robustness to Label Noise
Authors: Diego Ortego, Eric Arazo, Paul Albert, Noel E. O'Connor and Kevin McGuinness
Abstract summary: We show that standard supervised contrastive learning degrades in the presence of label noise. We propose a novel label noise detection method that exploits the robust feature representations learned via contrastive learning. Experiments on synthetic and real-world noise benchmarks demonstrate that MOIT/MOIT+ achieves state-of-the-art results.
Score: 17.264550056296915
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Deep neural networks trained with standard cross-entropy loss memorize noisy labels, which degrades their performance. Most research to mitigate this memorization proposes new robust classification loss functions. Conversely, we propose a Multi-Objective Interpolation Training (MOIT) approach that jointly exploits contrastive learning and classification to mutually help each other and boost performance against label noise. We show that standard supervised contrastive learning degrades in the presence of label noise and propose an interpolation training strategy to mitigate this behavior. We further propose a novel label noise detection method that exploits the robust feature representations learned via contrastive learning to estimate per-sample soft-labels whose disagreements with the original labels accurately identify noisy samples. This detection allows treating noisy samples as unlabeled and training a classifier in a semi-supervised manner to prevent noise memorization and improve representation learning. We further propose MOIT+, a refinement of MOIT by fine-tuning on detected clean samples. Hyperparameter and ablation studies verify the key components of our method. Experiments on synthetic and real-world noise benchmarks demonstrate that MOIT/MOIT+ achieves state-of-the-art results. Code is available at https://git.io/JI40X.