Exploring the Efficacy of Group-Normalization in Deep Learning Models for Alzheimer's Disease Classification

• URL: http://arxiv.org/abs/2404.00946v1
• Date: Mon, 1 Apr 2024 06:10:11 GMT
• Title: Exploring the Efficacy of Group-Normalization in Deep Learning Models for Alzheimer's Disease Classification
• Authors: Gousia Habib, Ishfaq Ahmed Malik, Jameel Ahmad, Imtiaz Ahmed, Shaima Qureshi,
• Abstract summary: Group Normalization is an easy alternative to Batch Normalization. GN achieves a very low error rate of 10.6% compared to Batch Normalization.
• Score: 2.6447365674762273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Batch Normalization is an important approach to advancing deep learning since it allows multiple networks to train simultaneously. A problem arises when normalizing along the batch dimension because B.N.'s error increases significantly as batch size shrinks because batch statistics estimates are inaccurate. As a result, computer vision tasks like detection, segmentation, and video, which require tiny batches based on memory consumption, aren't suitable for using Batch Normalization for larger model training and feature transfer. Here, we explore Group Normalization as an easy alternative to using Batch Normalization A Group Normalization is a channel normalization method in which each group is divided into different channels, and the corresponding mean and variance are calculated for each group. Group Normalization computations are accurate across a wide range of batch sizes and are independent of batch size. When trained using a large ImageNet database on ResNet-50, GN achieves a very low error rate of 10.6% compared to Batch Normalization. when a smaller batch size of only 2 is used. For usual batch sizes, the performance of G.N. is comparable to that of Batch Normalization, but at the same time, it outperforms other normalization techniques. Implementing Group Normalization as a direct alternative to B.N to combat the serious challenges faced by the Batch Normalization in deep learning models with comparable or improved classification accuracy. Additionally, Group Normalization can be naturally transferred from the pre-training to the fine-tuning phase. .

Related papers

• The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks [117.93273337740442]
  We show that gradient descent converges to a uniform margin classifier on the training data with an $exp(-Omega(log2 t))$ convergence rate. We also show that batch normalization has an implicit bias towards a patch-wise uniform margin.
  arXiv  Detail & Related papers  (2023-06-20T16:58:00Z)
• Context Normalization Layer with Applications [0.1499944454332829]
  This study proposes a new normalization technique, called context normalization, for image data. It adjusts the scaling of features based on the characteristics of each sample, which improves the model's convergence speed and performance. The effectiveness of context normalization is demonstrated on various datasets, and its performance is compared to other standard normalization techniques.
  arXiv  Detail & Related papers  (2023-03-14T06:38:17Z)
• Zero-Shot Anomaly Detection via Batch Normalization [58.291409630995744]
  Anomaly detection plays a crucial role in many safety-critical application domains. The challenge of adapting an anomaly detector to drift in the normal data distribution has led to the development of zero-shot AD techniques. We propose a simple yet effective method called Adaptive Centered Representations (ACR) for zero-shot batch-level AD.
  arXiv  Detail & Related papers  (2023-02-15T18:34:15Z)
• Compound Batch Normalization for Long-tailed Image Classification [77.42829178064807]
  We propose a compound batch normalization method based on a Gaussian mixture. It can model the feature space more comprehensively and reduce the dominance of head classes. The proposed method outperforms existing methods on long-tailed image classification.
  arXiv  Detail & Related papers  (2022-12-02T07:31:39Z)
• Batch Layer Normalization, A new normalization layer for CNNs and RNN [0.0]
  This study introduces a new normalization layer termed Batch Layer Normalization (BLN) As a combined version of batch and layer normalization, BLN adaptively puts appropriate weight on mini-batch and feature normalization based on the inverse size of mini-batches. Test results indicate the application potential of BLN and its faster convergence than batch normalization and layer normalization in both Convolutional and Recurrent Neural Networks.
  arXiv  Detail & Related papers  (2022-09-19T10:12:51Z)
• Batch Group Normalization [45.03388237812212]
  Batch Normalization (BN) performs well at medium and large batch sizes. BN saturates at small/extreme large batch sizes due to noisy/confused statistic calculation. BGN is proposed to solve the noisy/confused statistic calculation of BN at small/extreme large batch sizes.
  arXiv  Detail & Related papers  (2020-12-04T18:57:52Z)
• Comparing Normalization Methods for Limited Batch Size Segmentation Neural Networks [0.0]
  Batch Normalization works best using large batch size during training. We show the effectiveness of Instance Normalization in the limited batch size neural network training environment. We also show that the Instance Normalization implementation used in this experiment is computational time efficient when compared to the network without any normalization method.
  arXiv  Detail & Related papers  (2020-11-23T17:13:24Z)
• Double Forward Propagation for Memorized Batch Normalization [68.34268180871416]
  Batch Normalization (BN) has been a standard component in designing deep neural networks (DNNs) We propose a memorized batch normalization (MBN) which considers multiple recent batches to obtain more accurate and robust statistics. Compared to related methods, the proposed MBN exhibits consistent behaviors in both training and inference.
  arXiv  Detail & Related papers  (2020-10-10T08:48:41Z)
• Extended Batch Normalization [3.377000738091241]
  Batch normalization (BN) has become a standard technique for training the modern deep networks. In this paper, we propose a simple but effective method, called extended batch normalization (EBN) Experiments show that extended batch normalization alleviates the problem of batch normalization with small batch size while achieving close performances to batch normalization with large batch size.
  arXiv  Detail & Related papers  (2020-03-12T01:53:15Z)
• Cross-Iteration Batch Normalization [67.83430009388678]
  We present Cross-It Batch Normalization (CBN), in which examples from multiple recent iterations are jointly utilized to enhance estimation quality. CBN is found to outperform the original batch normalization and a direct calculation of statistics over previous iterations without the proposed compensation technique.
  arXiv  Detail & Related papers  (2020-02-13T18:52:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.