Effect of Activation Function and Model Optimizer on the Performance of Human Activity Recognition System Using Various Deep Learning Models

• URL: http://arxiv.org/abs/2512.20104v1
• Date: Tue, 23 Dec 2025 07:01:45 GMT
• Title: Effect of Activation Function and Model Optimizer on the Performance of Human Activity Recognition System Using Various Deep Learning Models
• Authors: Subrata Kumer Paula, Dewan Nafiul Islam Noora, Rakhi Rani Paula, Md. Ekramul Hamidb, Fahmid Al Faridc, Hezerul Abdul Karimd, Md. Maruf Al Hossain Princee, Abu Saleh Musa Miahb,
• Abstract summary: We investigate the effect of three commonly used activation functions (ReLU, Sigmoid, and Tanh) combined with four optimization algorithms.<n>Experiments are conducted on six medically relevant activity classes selected from the HMDB51 and UCF101 datasets.<n>Results show that ConvLSTM consistently outperforms BiLSTM across both datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human Activity Recognition (HAR) plays a vital role in healthcare, surveillance, and innovative environments, where reliable action recognition supports timely decision-making and automation. Although deep learning-based HAR systems are widely adopted, the impact of Activation Functions (AFs) and Model Optimizers (MOs) on performance has not been sufficiently analyzed, particularly regarding how their combinations influence model behavior in practical scenarios. Most existing studies focus on architecture design, while the interaction between AF and MO choices remains relatively unexplored. In this work, we investigate the effect of three commonly used activation functions (ReLU, Sigmoid, and Tanh) combined with four optimization algorithms (SGD, Adam, RMSprop, and Adagrad) using two recurrent deep learning architectures, namely BiLSTM and ConvLSTM. Experiments are conducted on six medically relevant activity classes selected from the HMDB51 and UCF101 datasets, considering their suitability for healthcare-oriented HAR applications. Our experimental results show that ConvLSTM consistently outperforms BiLSTM across both datasets. ConvLSTM, combined with Adam or RMSprop, achieves an accuracy of up to 99.00%, demonstrating strong spatio-temporal learning capabilities and stable performance. While BiLSTM performs reasonably well on UCF101, with accuracy approaching 98.00%, its performance drops to approximately 60.00% on HMDB51, indicating limited robustness across datasets and weaker sensitivity to AF and MO variations. This study provides practical insights for optimizing HAR systems, particularly for real-world healthcare environments where fast and precise activity detection is critical.

Related papers

• Studying Various Activation Functions and Non-IID Data for Machine Learning Model Robustness [3.641683644638084]
  We study the machine learning (ML) model robustness using ten different activation functions through adversarial training.<n>Our proposed centralized adversarial training approach achieves a natural and robust accuracy of 77.08% and 67.96%.<n>In the federated learning environment, however, the robust accuracy decreases significantly, especially on non-IID data.
  arXiv  Detail & Related papers  (2025-12-03T21:03:45Z)
• FAIM: Frequency-Aware Interactive Mamba for Time Series Classification [87.84511960413715]
  Time series classification (TSC) is crucial in numerous real-world applications, such as environmental monitoring, medical diagnosis, and posture recognition.<n>We propose FAIM, a lightweight Frequency-Aware Interactive Mamba model.<n>We show that FAIM consistently outperforms existing state-of-the-art (SOTA) methods, achieving a superior trade-off between accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-11-26T08:36:33Z)
• DeepACTIF: Efficient Feature Attribution via Activation Traces in Neural Sequence Models [0.0]
  Feature attribution is essential for interpreting deep learning models in time-series domains such as healthcare, biometrics, and human-AI interaction.<n>Standard attribution methods, such as Integrated Gradients or SHAP, are computationally intensive and not well-suited for real-time applications.<n>We present DeepACTIF, a lightweight and architecture-aware feature attribution method that leverages internal activations of sequence models to estimate feature importance efficiently.
  arXiv  Detail & Related papers  (2025-09-18T15:47:05Z)
• Toward Improving fNIRS Classification: A Study on Activation Functions in Deep Neural Architectures [7.243563999211656]
  Activation functions are critical to the performance of deep neural networks in domains such as functional near-infrared spectroscopy (fNIRS)<n>This study evaluates a range of conventional and field-specific activation functions for fNIRS classification tasks using multiple deep learning architectures.
  arXiv  Detail & Related papers  (2025-07-15T15:58:36Z)
• Evaluating BiLSTM and CNN+GRU Approaches for Human Activity Recognition Using WiFi CSI Data [0.0]
  CNN+GRU model has a higher accuracy on the UT-HAR dataset (95.20%) thanks to its ability to extract spatial features.<n>BiLSTM model performs better on the high-resolution NTU-Fi HAR dataset (92.05%) by extracting long-term temporal dependencies.<n>We show the real-world applicability of such models in applications like healthcare and intelligent home systems.
  arXiv  Detail & Related papers  (2025-06-11T22:10:34Z)
• Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes [70.66864668709677]
  We consider the problem of active learning for global sensitivity analysis of expensive black-box functions. Since function evaluations are expensive, we use active learning to prioritize experimental resources where they yield the most value. We propose novel active learning acquisition functions that directly target key quantities of derivative-based global sensitivity measures.
  arXiv  Detail & Related papers  (2024-07-13T01:41:12Z)
• APALU: A Trainable, Adaptive Activation Function for Deep Learning Networks [0.7919810878571297]
  We introduce a novel trainable activation function, adaptive piecewise approximated activation linear unit (APALU)<n> Experiments reveal significant improvements over widely used activation functions for different tasks.<n>APALU achieves 100% accuracy on a sign language recognition task with a limited dataset.
  arXiv  Detail & Related papers  (2024-02-13T06:18:42Z)
• Physics Inspired Hybrid Attention for SAR Target Recognition [61.01086031364307]
  We propose a physics inspired hybrid attention (PIHA) mechanism and the once-for-all (OFA) evaluation protocol to address the issues. PIHA leverages the high-level semantics of physical information to activate and guide the feature group aware of local semantics of target. Our method outperforms other state-of-the-art approaches in 12 test scenarios with same ASC parameters.
  arXiv  Detail & Related papers  (2023-09-27T14:39:41Z)
• Learning Objective-Specific Active Learning Strategies with Attentive Neural Processes [72.75421975804132]
  Learning Active Learning (LAL) suggests to learn the active learning strategy itself, allowing it to adapt to the given setting. We propose a novel LAL method for classification that exploits symmetry and independence properties of the active learning problem. Our approach is based on learning from a myopic oracle, which gives our model the ability to adapt to non-standard objectives.
  arXiv  Detail & Related papers  (2023-09-11T14:16:37Z)
• Shared Space Transfer Learning for analyzing multi-site fMRI data [83.41324371491774]
  Multi-voxel pattern analysis (MVPA) learns predictive models from task-based functional magnetic resonance imaging (fMRI) data. MVPA works best with a well-designed feature set and an adequate sample size. Most fMRI datasets are noisy, high-dimensional, expensive to collect, and with small sample sizes. This paper proposes the Shared Space Transfer Learning (SSTL) as a novel transfer learning approach.
  arXiv  Detail & Related papers  (2020-10-24T08:50:26Z)
• Learn to cycle: Time-consistent feature discovery for action recognition [83.43682368129072]
  Generalizing over temporal variations is a prerequisite for effective action recognition in videos. We introduce Squeeze Re Temporal Gates (SRTG), an approach that favors temporal activations with potential variations. We show consistent improvement when using SRTPG blocks, with only a minimal increase in the number of GFLOs.
  arXiv  Detail & Related papers  (2020-06-15T09:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.