Adaptive Gaussian Process Search for Simulation-Based Sample Size Estimation in Clinical Prediction Models: Validation of the pmsims R Package

• URL: http://arxiv.org/abs/2603.23688v1
• Date: Tue, 24 Mar 2026 19:53:37 GMT
• Title: Adaptive Gaussian Process Search for Simulation-Based Sample Size Estimation in Clinical Prediction Models: Validation of the pmsims R Package
• Abstract summary: We present and validate pmsims, an R package that uses Gaussian process surrogate modelling to provide a flexible simulation-based framework for sample size determination.<n>We benchmarked the best-performing pmsims engine against existing analytical (pmsampsize) and simulation-based (samplesizedev) methods.
• Score: 0.45393070061887714
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: Determining an adequate sample size is essential for developing reliable and generalisable clinical prediction models, yet practical guidance on selecting appropriate methods remains limited. Existing analytical and simulation-based approaches often rely on restrictive assumptions and focus on mean-based criteria. We present and validate pmsims, an R package that uses Gaussian process surrogate modelling to provide a flexible and computationally efficient simulation-based framework for sample size determination across diverse prediction settings. Methods: We conducted a comprehensive simulation study with two aims. First, we compared three search engines implemented in pmsims: a Gaussian process-based adaptive method, a deterministic bisection method, and a hybrid approach, across binary, continuous, and survival outcomes. Second, we benchmarked the best-performing pmsims engine against existing analytical (pmsampsize) and simulation-based (samplesizedev) methods, evaluating recommended sample sizes, computational time, and achieved performance on large independent validation datasets. Results: The Gaussian process-based method consistently produced the most stable sample size estimates, particularly in low-signal, high-dimensional settings. In benchmarking, pmsims achieved performance close to prespecified targets across all outcome types, matching simulation-based approaches and outperforming analytical methods in more challenging scenarios. Conclusions: pmsims provides an efficient and flexible framework for principled sample size planning in clinical prediction modelling, requiring fewer model evaluations than non-adaptive simulation approaches.

Related papers

• Hybrid Cold-Start Recommender System for Closure Model Selection in Multiphase Flow Simulations [0.0]
  This work formulates closure model selection as a cold-start recommender system problem in a high-cost scientific domain.<n>We propose a hybrid recommendation framework that combines (i) metadata-driven case similarity and (ii) collaborative inference via matrix completion.<n>The methodology is evaluated on 13,600 simulations across 136 validation cases and 100 model combinations.
  arXiv  Detail & Related papers  (2026-04-10T08:46:30Z)
• Sample Size Calculations for Developing Clinical Prediction Models: Overview and pmsims R package [0.45393070061887714]
  Inadequate sample sizes can lead to overfitting, poor generalisability, and biased predictions.<n>Existing approaches, such as rules, closed-form formulas, and simulation-based methods, vary in flexibility and accuracy.<n>We introduce a conceptual framework that distinguishes between mean-based and assurance-based criteria.<n>We propose a novel simulation-based approach that integrates learning curves, Gaussian Process optimisation, and assurance principles.
  arXiv  Detail & Related papers  (2026-02-26T21:24:34Z)
• Machine learning to optimize precision in the analysis of randomized trials: A journey in pre-specified, yet data-adaptive learning [2.6827221447298406]
  We tell our story of developing, evaluating, and implementing a machine learning-based approach for covariate adjustment.<n>We provide the rationale for as well as the practical concerns with such an approach for estimating marginal effects.<n>We present the results from applying our approach in the primary, pre-specified analysis of 8 recently published trials.
  arXiv  Detail & Related papers  (2025-12-15T18:05:45Z)
• Simulation-based Methods for Optimal Sampling Design in Systems Biology [0.38233569758620045]
  This study proposes two simulation-based methods for optimal sampling design.<n>The first method, E-optimal-ranking (EOR), employs the E-optimal criterion, while the second utilizes a Long Short-Term Memory (LSTM) neural network.<n> Simulation studies based on the Lotka-Volterra and three-compartment models demonstrate that the proposed methods outperform both random selection and classical E-optimal design.
  arXiv  Detail & Related papers  (2025-11-10T15:26:51Z)
• Model Correlation Detection via Random Selection Probing [62.093777777813756]
  Existing similarity-based methods require access to model parameters or produce scores without thresholds.<n>We introduce Random Selection Probing (RSP), a hypothesis-testing framework that formulates model correlation detection as a statistical test.<n>RSP produces rigorous p-values that quantify evidence of correlation.
  arXiv  Detail & Related papers  (2025-09-29T01:40:26Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective.<n>The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning.<n>The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• An incremental preference elicitation-based approach to learning potentially non-monotonic preferences in multi-criteria sorting [53.36437745983783]
  We first construct a max-margin optimization-based model to model potentially non-monotonic preferences. We devise information amount measurement methods and question selection strategies to pinpoint the most informative alternative in each iteration. Two incremental preference elicitation-based algorithms are developed to learn potentially non-monotonic preferences.
  arXiv  Detail & Related papers  (2024-09-04T14:36:20Z)
• Conformal Approach To Gaussian Process Surrogate Evaluation With Coverage Guarantees [47.22930583160043]
  We propose a method for building adaptive cross-conformal prediction intervals. The resulting conformal prediction intervals exhibit a level of adaptivity akin to Bayesian credibility sets. The potential applicability of the method is demonstrated in the context of surrogate modeling of an expensive-to-evaluate simulator of the clogging phenomenon in steam generators of nuclear reactors.
  arXiv  Detail & Related papers  (2024-01-15T14:45:18Z)
• Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference [44.281860162298564]
  We introduce two synthetic likelihood methods for Simulation-Based Inference. We learn a conditional energy-based model (EBM) of the likelihood using synthetic data generated by the simulator. We demonstrate the properties of both methods on a range of synthetic datasets, and apply them to a model of the neuroscience network in the crab.
  arXiv  Detail & Related papers  (2022-10-26T14:38:24Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• fETSmcs: Feature-based ETS model component selection [8.99236558175168]
  We propose an efficient approach for ETS model selection by training classifiers on simulated data to predict appropriate model component forms for a given time series. We evaluate our approach on the widely used forecasting competition data set M4 in terms of both point forecasts and prediction intervals.
  arXiv  Detail & Related papers  (2022-06-26T13:52:43Z)
• Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set [3.862247454265944]
  We develop a framework to develop calibration schemes that satisfy rigorous frequentist statistical guarantees. We demonstrate our methodology on several numerical examples, including an application to calibration of a limit order book market simulator.
  arXiv  Detail & Related papers  (2021-05-27T00:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.