A novel three-stage training strategy for long-tailed classification
- URL: http://arxiv.org/abs/2104.09830v1
- Date: Tue, 20 Apr 2021 08:29:27 GMT
- Title: A novel three-stage training strategy for long-tailed classification
- Authors: Gongzhe Li, Zhiwen Tan, Linpeng Pan
- Abstract summary: Long-tailed distribution datasets pose great challenges for deep learning based classification models.
We establish a novel three-stages training strategy, which has excellent results for processing SAR image datasets with long-tailed distribution.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: The long-tailed distribution datasets poses great challenges for deep
learning based classification models on how to handle the class imbalance
problem. Existing solutions usually involve class-balacing strategies or
transfer learing from head- to tail-classes or use two-stages learning strategy
to re-train the classifier. However, the existing methods are difficult to
solve the low quality problem when images are obtained by SAR. To address this
problem, we establish a novel three-stages training strategy, which has
excellent results for processing SAR image datasets with long-tailed
distribution. Specifically, we divide training procedure into three stages. The
first stage is to use all kinds of images for rough-training, so as to get the
rough-training model with rich content. The second stage is to make the rough
model learn the feature expression by using the residual dataset with the class
0 removed. The third stage is to fine tune the model using class-balanced
datasets with all 10 classes (including the overall model fine tuning and
classifier re-optimization). Through this new training strategy, we only use
the information of SAR image dataset and the network model with very small
parameters to achieve the top 1 accuracy of 22.34 in development phase.
Related papers
- Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud
Semantic Segmentation via Decoupling Optimization [64.36097398869774]
Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding.
The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data.
We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
arXiv Detail & Related papers (2024-01-13T04:16:40Z) - Two-Stage Triplet Loss Training with Curriculum Augmentation for
Audio-Visual Retrieval [3.164991885881342]
Cross- retrieval models learn robust embedding spaces.
We introduce a novel approach rooted in curriculum learning to address this problem.
We propose a two-stage training paradigm that guides the model's learning process from semi-hard to hard triplets.
arXiv Detail & Related papers (2023-10-20T12:35:54Z) - No Data Augmentation? Alternative Regularizations for Effective Training
on Small Datasets [0.0]
We study alternative regularization strategies to push the limits of supervised learning on small image classification datasets.
In particular, we employ a agnostic to select (semi) optimal learning rate and weight decay couples via the norm of model parameters.
We reach a test accuracy of 66.5%, on par with the best state-of-the-art methods.
arXiv Detail & Related papers (2023-09-04T16:13:59Z) - Neural Collapse Terminus: A Unified Solution for Class Incremental
Learning and Its Variants [166.916517335816]
In this paper, we offer a unified solution to the misalignment dilemma in the three tasks.
We propose neural collapse terminus that is a fixed structure with the maximal equiangular inter-class separation for the whole label space.
Our method holds the neural collapse optimality in an incremental fashion regardless of data imbalance or data scarcity.
arXiv Detail & Related papers (2023-08-03T13:09:59Z) - MIS-FM: 3D Medical Image Segmentation using Foundation Models Pretrained
on a Large-Scale Unannotated Dataset [14.823114726604853]
We propose a novel self-supervised learning strategy named Volume Fusion (VF) for pretraining 3D segmentation models.
VF forces the model to predict the fusion coefficient of each voxel, which is formulated as a self-supervised segmentation task without manual annotations.
experiments with different downstream segmentation targets including head and neck organs, thoracic/abdominal organs showed that our pretrained model largely outperformed training from scratch.
arXiv Detail & Related papers (2023-06-29T13:22:13Z) - Boosting Visual-Language Models by Exploiting Hard Samples [126.35125029639168]
HELIP is a cost-effective strategy tailored to enhance the performance of existing CLIP models.
Our method allows for effortless integration with existing models' training pipelines.
On comprehensive benchmarks, HELIP consistently boosts existing models to achieve leading performance.
arXiv Detail & Related papers (2023-05-09T07:00:17Z) - Two-Stage Fine-Tuning: A Novel Strategy for Learning Class-Imbalanced
Data [11.66734752179563]
Classification on long-tailed distributed data is a challenging problem.
Learning on tail classes is especially challenging for the fine-tuning when transferring a pretrained model to a downstream task.
We propose a two-stage fine-tuning: we first fine-tune the final layer of the pretrained model with class-balanced reweighting loss, and then we perform the standard fine-tuning.
arXiv Detail & Related papers (2022-07-22T03:39:51Z) - Adaptive Fine-Grained Sketch-Based Image Retrieval [100.90633284767205]
Recent focus on Fine-Grained Sketch-Based Image Retrieval has shifted towards generalising a model to new categories.
In real-world applications, a trained FG-SBIR model is often applied to both new categories and different human sketchers.
We introduce a novel model-agnostic meta-learning (MAML) based framework with several key modifications.
arXiv Detail & Related papers (2022-07-04T21:07:20Z) - Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV)
NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones.
We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
arXiv Detail & Related papers (2022-06-23T09:35:03Z) - Set Based Stochastic Subsampling [85.5331107565578]
We propose a set-based two-stage end-to-end neural subsampling model that is jointly optimized with an textitarbitrary downstream task network.
We show that it outperforms the relevant baselines under low subsampling rates on a variety of tasks including image classification, image reconstruction, function reconstruction and few-shot classification.
arXiv Detail & Related papers (2020-06-25T07:36:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.