A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture

• URL: http://arxiv.org/abs/2506.15737v1
• Date: Tue, 17 Jun 2025 04:12:58 GMT
• Title: A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture
• Authors: Gautam Siddharth Kashyap, Md Tabrez Nafis, Samar Wazir,
• Abstract summary: Training Artificial Neural Networks (ANNs) with Gradient Descent (SGD) frequently difficulties encounters.<n>This work investigates Particle Swarm Optimization (PSO) and Genetic Algorithms (GAs)<n>A hybrid PSO-SGD strategy is developed to improve local search efficiency.
• Score: 1.024113475677323
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Training Artificial Neural Networks (ANNs) with Stochastic Gradient Descent (SGD) frequently encounters difficulties, including substantial computing expense and the risk of converging to local optima, attributable to its dependence on partial weight gradients. Therefore, this work investigates Particle Swarm Optimization (PSO) and Genetic Algorithms (GAs) - two population-based Metaheuristic Optimizers (MHOs) - as alternatives to SGD to mitigate these constraints. A hybrid PSO-SGD strategy is developed to improve local search efficiency. The findings indicate that the hybrid PSO-SGD technique decreases the median training MSE by 90 to 95 percent relative to conventional GA and PSO across various network sizes (e.g., from around 0.02 to approximately 0.001 in the Sphere function). RMHC attains substantial enhancements, reducing MSE by roughly 85 to 90 percent compared to GA. Simultaneously, RS consistently exhibits errors exceeding 0.3, signifying subpar performance. These findings underscore that hybrid and evolutionary procedures significantly improve training efficiency and accuracy compared to conventional optimization methods and imply that the Building Block Hypothesis (BBH) may still be valid, indicating that advantageous weight structures are retained during evolutionary search.

Related papers

• Integrating Complexity and Biological Realism: High-Performance Spiking Neural Networks for Breast Cancer Detection [0.0]
  Spiking Neural Networks (SNNs) event-driven nature enables efficient encoding of spatial and temporal features.<n>SNNs have seen limited application in medical image recognition due to difficulties in matching the performance of conventional deep learning models.<n>We propose a novel breast cancer classification approach that combines SNNs with Lempel-Ziv Complexity (LZC) a computationally efficient measure of sequence complexity.
  arXiv  Detail & Related papers  (2025-06-06T17:47:27Z)
• PSO-UNet: Particle Swarm-Optimized U-Net Framework for Precise Multimodal Brain Tumor Segmentation [0.0]
  This study introduces PSO-UNet, which integrates Particle Swarm Optimization (PSO) with the U-Net architecture for dynamic hyper parameter optimization.<n>PSO-UNet substantially enhances segmentation performance, achieving Dice Similarity Coefficients (DSC) of 0.9578 and 0.9523 on the BraTS 2021 and Figshare datasets, respectively.<n>The method reduces computational complexity significantly, utilizing only 7.8 million parameters and executing in approximately 906 seconds, markedly faster than comparable U-Net-based frameworks.
  arXiv  Detail & Related papers  (2025-03-24T21:14:08Z)
• Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks [0.0]
  The research presented in this paper advances the integration of Hebbian learning into Convolutional Neural Networks (CNNs) for image processing.<n>Hebbian learning operates on local unsupervised neural information to form feature representations.<n>Results showed clear indications of sparse hierarchical learning through increasingly complex and receptive fields.
  arXiv  Detail & Related papers  (2025-01-06T12:29:37Z)
• Stability and Generalization for Distributed SGDA [70.97400503482353]
  We propose the stability-based generalization analytical framework for Distributed-SGDA. We conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics. Our theoretical results reveal the trade-off between the generalization gap and optimization error.
  arXiv  Detail & Related papers  (2024-11-14T11:16:32Z)
• An Effective Networks Intrusion Detection Approach Based on Hybrid Harris Hawks and Multi-Layer Perceptron [47.81867479735455]
  This paper proposes an Intrusion Detection System (IDS) employing the Harris Hawks Optimization (HHO) to optimize Multilayer Perceptron learning. HHO-MLP aims to select optimal parameters in its learning process to minimize intrusion detection errors in networks. HHO-MLP showed superior performance by attaining top scores with accuracy rate of 93.17%, sensitivity level of 95.41%, and specificity percentage of 95.41%.
  arXiv  Detail & Related papers  (2024-02-21T06:25:50Z)
• Membrane Potential Distribution Adjustment and Parametric Surrogate Gradient in Spiking Neural Networks [3.485537704990941]
  Surrogate gradient (SG) strategy is investigated and applied to circumvent this issue and train SNNs from scratch. We propose the parametric surrogate gradient (PSG) method to iteratively update SG and eventually determine an optimal surrogate gradient parameter. Experimental results demonstrate that the proposed methods can be readily integrated with backpropagation through time (BPTT) algorithm.
  arXiv  Detail & Related papers  (2023-04-26T05:02:41Z)
• Reinforced Genetic Algorithm for Structure-based Drug Design [38.134929249388406]
  Structure-based drug design (SBDD) aims to discover drug candidates by finding molecules that bind to a disease-related protein (targets) We propose Reinforced Genetic Algorithm (RGA) that uses neural models to prioritize the profitable design steps and suppress random-walk behavior.
  arXiv  Detail & Related papers  (2022-11-28T22:59:46Z)
• Dissecting adaptive methods in GANs [46.90376306847234]
  We study how adaptive methods help train generative adversarial networks (GANs) By considering an update rule with the magnitude of the Adam update and the normalized direction of SGD, we empirically show that the adaptive magnitude of Adam is key for GAN training. We prove that in that setting, GANs trained with nSGDA recover all the modes of the true distribution, whereas the same networks trained with SGDA (and any learning rate configuration) suffer from mode collapse.
  arXiv  Detail & Related papers  (2022-10-09T19:00:07Z)
• Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality [65.67315418971688]
  Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR) are proposed. Experiments on visual recognition demonstrate that our methods can simultaneously improve the covariance conditioning and generalization.
  arXiv  Detail & Related papers  (2022-07-05T15:39:29Z)
• Free-form Lesion Synthesis Using a Partial Convolution Generative Adversarial Network for Enhanced Deep Learning Liver Tumor Segmentation [3.3148826359547523]
  This study aims to develop a deep learning framework for generating synthetic lesions that can be used to enhance network training. The lesion synthesis network is a modified generative adversarial network (GAN) The discriminator is designed using Wasserstein GAN with gradient penalty and spectral normalization.
  arXiv  Detail & Related papers  (2022-06-18T00:40:41Z)
• Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data [125.7135706352493]
  Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator.
  arXiv  Detail & Related papers  (2021-11-12T18:13:45Z)
• Convergence Analysis of Homotopy-SGD for non-convex optimization [43.71213126039448]
  We present a first-order algorithm based on a combination of homotopy methods and SGD, called Gradienty-Stoch Descent (H-SGD) Under some assumptions, we conduct a theoretical analysis of the proposed problem. Experimental results show that H-SGD can outperform SGD.
  arXiv  Detail & Related papers  (2020-11-20T09:50:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.