Compound Batch Normalization for Long-tailed Image Classification
- URL: http://arxiv.org/abs/2212.01007v1
- Date: Fri, 2 Dec 2022 07:31:39 GMT
- Title: Compound Batch Normalization for Long-tailed Image Classification
- Authors: Lechao Cheng, Chaowei Fang, Dingwen Zhang, Guanbin Li, Gang Huang
- Abstract summary: We propose a compound batch normalization method based on a Gaussian mixture.
It can model the feature space more comprehensively and reduce the dominance of head classes.
The proposed method outperforms existing methods on long-tailed image classification.
- Score: 77.42829178064807
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Significant progress has been made in learning image classification neural
networks under long-tail data distribution using robust training algorithms
such as data re-sampling, re-weighting, and margin adjustment. Those methods,
however, ignore the impact of data imbalance on feature normalization. The
dominance of majority classes (head classes) in estimating statistics and
affine parameters causes internal covariate shifts within less-frequent
categories to be overlooked. To alleviate this challenge, we propose a compound
batch normalization method based on a Gaussian mixture. It can model the
feature space more comprehensively and reduce the dominance of head classes. In
addition, a moving average-based expectation maximization (EM) algorithm is
employed to estimate the statistical parameters of multiple Gaussian
distributions. However, the EM algorithm is sensitive to initialization and can
easily become stuck in local minima where the multiple Gaussian components
continue to focus on majority classes. To tackle this issue, we developed a
dual-path learning framework that employs class-aware split feature
normalization to diversify the estimated Gaussian distributions, allowing the
Gaussian components to fit with training samples of less-frequent classes more
comprehensively. Extensive experiments on commonly used datasets demonstrated
that the proposed method outperforms existing methods on long-tailed image
classification.
Related papers
- Probabilistic Contrastive Learning for Long-Tailed Visual Recognition [78.70453964041718]
Longtailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples.
Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance.
We propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space.
arXiv Detail & Related papers (2024-03-11T13:44:49Z) - Sparse Variational Contaminated Noise Gaussian Process Regression with Applications in Geomagnetic Perturbations Forecasting [4.675221539472143]
We propose a scalable inference algorithm for fitting sparse Gaussian process regression models with contaminated normal noise on large datasets.
We show that our approach yields shorter prediction intervals for similar coverage and accuracy when compared to an artificial dense neural network baseline.
arXiv Detail & Related papers (2024-02-27T15:08:57Z) - Intra-class Adaptive Augmentation with Neighbor Correction for Deep
Metric Learning [99.14132861655223]
We propose a novel intra-class adaptive augmentation (IAA) framework for deep metric learning.
We reasonably estimate intra-class variations for every class and generate adaptive synthetic samples to support hard samples mining.
Our method significantly improves and outperforms the state-of-the-art methods on retrieval performances by 3%-6%.
arXiv Detail & Related papers (2022-11-29T14:52:38Z) - Learnable Distribution Calibration for Few-Shot Class-Incremental
Learning [122.2241120474278]
Few-shot class-incremental learning (FSCIL) faces challenges of memorizing old class distributions and estimating new class distributions given few training samples.
We propose a learnable distribution calibration (LDC) approach, with the aim to systematically solve these two challenges using a unified framework.
arXiv Detail & Related papers (2022-10-01T09:40:26Z) - A New Robust Multivariate Mode Estimator for Eye-tracking Calibration [0.0]
We propose a new method for estimating the main mode of multivariate distributions, with application to eye-tracking calibrations.
In this type of multimodal distributions, most central tendency measures fail at estimating the principal fixation coordinates.
Here, we developed a new algorithm to identify the first mode of multivariate distributions, named BRIL.
We obtained outstanding performances, even for distributions containing very high proportions of outliers, both grouped in clusters and randomly distributed.
arXiv Detail & Related papers (2021-07-16T17:45:19Z) - Learning Gaussian Mixtures with Generalised Linear Models: Precise
Asymptotics in High-dimensions [79.35722941720734]
Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks.
We prove exacts characterising the estimator in high-dimensions via empirical risk minimisation.
We discuss how our theory can be applied beyond the scope of synthetic data.
arXiv Detail & Related papers (2021-06-07T16:53:56Z) - Scalable Marginal Likelihood Estimation for Model Selection in Deep
Learning [78.83598532168256]
Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties.
Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
arXiv Detail & Related papers (2021-04-11T09:50:24Z) - Binary Classification of Gaussian Mixtures: Abundance of Support
Vectors, Benign Overfitting and Regularization [39.35822033674126]
We study binary linear classification under a generative Gaussian mixture model.
We derive novel non-asymptotic bounds on the classification error of the latter.
Our results extend to a noisy model with constant probability noise flips.
arXiv Detail & Related papers (2020-11-18T07:59:55Z) - Handling missing data in model-based clustering [0.0]
We propose two methods to fit Gaussian mixtures in the presence of missing data.
Both methods use a variant of the Monte Carlo Expectation-Maximisation algorithm for data augmentation.
We show that the proposed methods outperform the multiple imputation approach, both in terms of clusters identification and density estimation.
arXiv Detail & Related papers (2020-06-04T15:36:31Z) - Minority Oversampling for Imbalanced Time Series Classification [7.695093197007146]
This paper proposes a structure preserving Oversampling method to combat the High-dimensional Imbalanced Time-series classification.
Experimental results on several publicly available time-series datasets demonstrate the superiority of OHIT against the state-of-the-art oversampling algorithms.
arXiv Detail & Related papers (2020-04-14T09:20:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.