Heart Disease Prediction: A Comparative Study of Optimisers Performance in Deep Neural Networks

• URL: http://arxiv.org/abs/2509.08499v1
• Date: Wed, 10 Sep 2025 11:15:44 GMT
• Title: Heart Disease Prediction: A Comparative Study of Optimisers Performance in Deep Neural Networks
• Authors: Chisom Chibuike, Adeyinka Ogunsanya,
• Abstract summary: We compare the performance of 10 different approaches in training a simple Multi-layer Perceptron model using a heart disease dataset from Kaggle.<n>We set up a consistent training paradigm and evaluate the metrics based on metrics such as convergence speed and stability.<n>Across all our metrics, we chose RMSProp to be the most effective for this heart disease prediction task because it offered a balanced performance across key metrics.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Optimization has been an important factor and topic of interest in training deep learning models, yet less attention has been given to how we select the optimizers we use to train these models. Hence, there is a need to dive deeper into how we select the optimizers we use for training and the metrics that determine this selection. In this work, we compare the performance of 10 different optimizers in training a simple Multi-layer Perceptron model using a heart disease dataset from Kaggle. We set up a consistent training paradigm and evaluate the optimizers based on metrics such as convergence speed and stability. We also include some other Machine Learning Evaluation metrics such as AUC, Precision, and Recall, which are central metrics to classification problems. Our results show that there are trade-offs between convergence speed and stability, as optimizers like Adagrad and Adadelta, which are more stable, took longer time to converge. Across all our metrics, we chose RMSProp to be the most effective optimizer for this heart disease prediction task because it offered a balanced performance across key metrics. It achieved a precision of 0.765, a recall of 0.827, and an AUC of 0.841, along with faster training time. However, it was not the most stable. We recommend that, in less compute-constrained environments, this method of choosing optimizers through a thorough evaluation should be adopted to increase the scientific nature and performance in training deep learning models.

Related papers

• Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales [55.91454326946738]
  We study how the optimal learning rate and weight decay should scale with model width and depth for a wide range of languages.<n>We find that scaling the learning rate according to $$P improves transfer, but can still suffer from significant finite-width deviations.<n>For compute-optimal scaling, we find scaling independent weight decay as $1/mathrmwidth$ is nearly optimal across languages.
  arXiv  Detail & Related papers  (2025-12-05T11:03:41Z)
• Fantastic Pretraining Optimizers and Where to Find Them [59.56075036649332]
  AdamW has long been the dominant gradients in language model pretraining.<n>Speedup of matrix-based matrices is inversely proportional to model scale.
  arXiv  Detail & Related papers  (2025-09-02T07:43:22Z)
• Optimizers Qualitatively Alter Solutions And We Should Leverage This [62.662640460717476]
  Deep Neural Networks (DNNs) can not guarantee convergence to a unique global minimum of the loss when using only local information, such as SGD.<n>We argue that the community should aim at understanding the biases of already existing methods, as well as aim to build new DNNs with the explicit intent of inducing certain properties of the solution.
  arXiv  Detail & Related papers  (2025-07-16T13:33:31Z)
• UniCBE: An Uniformity-driven Comparing Based Evaluation Framework with Unified Multi-Objective Optimization [19.673388630963807]
  We propose UniCBE, a unified uniformity-driven CBE framework.<n>On the AlpacaEval benchmark, UniCBE saves over 17% of evaluation budgets while achieving a Pearson correlation with ground truth exceeding 0.995.<n>In scenarios where new models are continuously introduced, UniCBE can even save over 50% of evaluation costs.
  arXiv  Detail & Related papers  (2025-02-17T05:28:12Z)
• Understanding Optimization in Deep Learning with Central Flows [53.66160508990508]
  We show that an RMS's implicit behavior can be explicitly captured by a "central flow:" a differential equation. We show that these flows can empirically predict long-term optimization trajectories of generic neural networks.
  arXiv  Detail & Related papers  (2024-10-31T17:58:13Z)
• CoRe Optimizer: An All-in-One Solution for Machine Learning [0.0]
  Continuously resilient convergence (CoRe) shown superior performance compared to other state-of-the-art first-order gradient-based convergence algorithms. CoRe yields best or competitive performance in every investigated application.
  arXiv  Detail & Related papers  (2023-07-28T16:48:42Z)
• VeLO: Training Versatile Learned Optimizers by Scaling Up [67.90237498659397]
  We leverage the same scaling approach behind the success of deep learning to learn versatiles. We train an ingest for deep learning which is itself a small neural network that ingests and outputs parameter updates. We open source our learned, meta-training code, the associated train test data, and an extensive benchmark suite with baselines at velo-code.io.
  arXiv  Detail & Related papers  (2022-11-17T18:39:07Z)
• A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases [44.01339030872185]
  Blackbox learneds often struggle with stability and generalization when applied to tasks unlike those in their meta-training set. We investigate the inductive biases and stability properties of optimization algorithms, and apply the resulting insights to designing inductive biases for blackboxs. We learn to a variety of neural network training tasks, where it outperforms the current state of the art learned.
  arXiv  Detail & Related papers  (2022-09-22T17:47:21Z)
• Bayesian Optimization for Selecting Efficient Machine Learning Models [53.202224677485525]
  We present a unified Bayesian Optimization framework for jointly optimizing models for both prediction effectiveness and training efficiency. Experiments on model selection for recommendation tasks indicate models selected this way significantly improves model training efficiency.
  arXiv  Detail & Related papers  (2020-08-02T02:56:30Z)
• FOAL: Fast Online Adaptive Learning for Cardiac Motion Estimation [20.642769236234848]
  Motion estimation of cardiac MRI videos is crucial for the evaluation of human heart anatomy and function. Recent researches show promising results with deep learning-based methods, but they suffer dramatic performance drops due to mismatched between training and testing datasets. We propose a novel fast online adaptive learning framework: an online gradient descent based that is optimized by a meta-learner.
  arXiv  Detail & Related papers  (2020-03-10T01:51:27Z)
• Self-Directed Online Machine Learning for Topology Optimization [58.920693413667216]
  Self-directed Online Learning Optimization integrates Deep Neural Network (DNN) with Finite Element Method (FEM) calculations. Our algorithm was tested by four types of problems including compliance minimization, fluid-structure optimization, heat transfer enhancement and truss optimization. It reduced the computational time by 2 5 orders of magnitude compared with directly using methods, and outperformed all state-of-the-art algorithms tested in our experiments.
  arXiv  Detail & Related papers  (2020-02-04T20:00:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.