Feature Importance Guided Random Forest Learning with Simulated Annealing Based Hyperparameter Tuning

• URL: http://arxiv.org/abs/2511.00133v1
• Date: Fri, 31 Oct 2025 12:14:53 GMT
• Title: Feature Importance Guided Random Forest Learning with Simulated Annealing Based Hyperparameter Tuning
• Authors: Kowshik Balasubramanian, Andre Williams, Ismail Butun,
• Abstract summary: This paper introduces a novel framework for enhancing Random Forest classifiers by integrating probabilistic feature sampling and hyper parameter tuning.<n>We tackle the multifaceted challenges of robust classification across diverse domains, including credit risk evaluation, anomaly detection in IoT ecosystems, early-stage medical diagnostics, and high-dimensional biological data analysis.<n>Results demonstrate consistent accuracy improvements and meaningful insights into feature relevance, showcasing the efficacy of combining importance aware sampling and metaheuristic optimization.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper introduces a novel framework for enhancing Random Forest classifiers by integrating probabilistic feature sampling and hyperparameter tuning via Simulated Annealing. The proposed framework exhibits substantial advancements in predictive accuracy and generalization, adeptly tackling the multifaceted challenges of robust classification across diverse domains, including credit risk evaluation, anomaly detection in IoT ecosystems, early-stage medical diagnostics, and high-dimensional biological data analysis. To overcome the limitations of conventional Random Forests, we present an approach that places stronger emphasis on capturing the most relevant signals from data while enabling adaptive hyperparameter configuration. The model is guided towards features that contribute more meaningfully to classification and optimizing this with dynamic parameter tuning. The results demonstrate consistent accuracy improvements and meaningful insights into feature relevance, showcasing the efficacy of combining importance aware sampling and metaheuristic optimization.

Related papers

• Generalizable Hyperparameter Optimization for Federated Learning on Non-IID Cancer Images [0.0]
  Deep learning for cancer histopathology training conflicts with privacy constraints in clinical settings.<n>Deep learning for cancer histopathology training conflicts with privacy constraints in clinical settings.<n> Federated Learning (FL) mitigates this by keeping data local.
  arXiv  Detail & Related papers  (2026-01-19T02:24:24Z)
• Optimizing Multi-Modal Trackers via Sensitivity-aware Regularized Tuning [112.12667472919723]
  This paper tackles the challenge of optimizing multi-modal trackers by effectively adapting the pre-trained models for RGB data.<n>Existing fine-tuning paradigms oscillate between excessive freedom and over-restriction, leading to a suboptimal plasticity-stability trade-off.<n>We propose a novel sensitivity-aware regularized tuning framework, which delicately refines the learning process by incorporating intrinsic parameter sensitivities.
  arXiv  Detail & Related papers  (2025-08-24T18:42:47Z)
• Robust Spatiotemporal Epidemic Modeling with Integrated Adaptive Outlier Detection [7.5504472850103435]
  In epidemic modeling, outliers can distort parameter estimation and lead to misguided public health decisions.<n>We introduce a robust generalized additive model (RST-GAM) to mitigate this distortion.<n>We demonstrate the practical utility of RST-GAM by analyzing county-level COVID-19 infection data in the United States.
  arXiv  Detail & Related papers  (2025-07-12T19:23:25Z)
• Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models [68.57424628540907]
  Large language models (LLMs) often develop learned mechanisms specialized to specific datasets.<n>We introduce a fine-tuning approach designed to enhance generalization by identifying and pruning neurons associated with dataset-specific mechanisms.<n>Our method employs Integrated Gradients to quantify each neuron's influence on high-confidence predictions, pinpointing those that disproportionately contribute to dataset-specific performance.
  arXiv  Detail & Related papers  (2025-07-12T08:10:10Z)
• Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios [54.58186816693791]
  environments constantly change over time and space, posing significant challenges for object detectors trained based on a closed-set assumption.<n>We propose a new mechanism, converting the fine-tuning process to a specific- parameter generation.<n>In particular, we first design a dual-path LoRA-based domain-aware adapter that disentangles features into domain-invariant and domain-specific components.
  arXiv  Detail & Related papers  (2025-06-30T17:14:12Z)
• The Role of Hyperparameters in Predictive Multiplicity [0.0]
  Different machine learning models trained on the same dataset yield divergent predictions for identical inputs.<n>These inconsistencies can seriously impact high-stakes decisions such as credit assessments, hiring, and medical diagnoses.
  arXiv  Detail & Related papers  (2025-03-13T19:22:44Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Subject-specific Deep Neural Networks for Count Data with High-cardinality Categorical Features [1.2289361708127877]
  We propose a novel hierarchical likelihood learning framework for introducing gamma random effects into a Poisson deep neural network. The proposed method simultaneously yields maximum likelihood estimators for fixed parameters and best unbiased predictors for random effects. State-of-the-art network architectures can be easily implemented into the proposed h-likelihood framework.
  arXiv  Detail & Related papers  (2023-10-18T01:54:48Z)
• Understanding Overparameterization in Generative Adversarial Networks [56.57403335510056]
  Generative Adversarial Networks (GANs) are used to train non- concave mini-max optimization problems. A theory has shown the importance of the gradient descent (GD) to globally optimal solutions. We show that in an overized GAN with a $1$-layer neural network generator and a linear discriminator, the GDA converges to a global saddle point of the underlying non- concave min-max problem.
  arXiv  Detail & Related papers  (2021-04-12T16:23:37Z)
• Selective Cascade of Residual ExtraTrees [3.6575928994425735]
  We propose a novel tree-based ensemble method named Selective Cascade of Residual ExtraTrees (SCORE) SCORE draws inspiration from representation learning, incorporates regularized regression with variable selection features, and utilizes boosting to improve prediction and reduce generalization errors. Our computer experiments show that SCORE provides comparable or superior performance in prediction against ExtraTrees, random forest, gradient boosting machine, and neural networks.
  arXiv  Detail & Related papers  (2020-09-29T16:31:37Z)
• An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization [48.5614138038673]
  We propose an efficient and robust bandit-based algorithm called Sub-Sampling (SS) in the scenario of hyper parameter search evaluation. We also develop a novel hyper parameter optimization algorithm called BOSS. Empirical studies validate our theoretical arguments of SS and demonstrate the superior performance of BOSS on a number of applications.
  arXiv  Detail & Related papers  (2020-07-11T03:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.