From Prediction to Practice: A Task-Aware Evaluation Framework for Blood Glucose Forecasting

• URL: http://arxiv.org/abs/2605.00645v1
• Date: Fri, 01 May 2026 13:26:28 GMT
• Title: From Prediction to Practice: A Task-Aware Evaluation Framework for Blood Glucose Forecasting
• Abstract summary: We present a task-aware evaluation framework for blood glucose forecasting built around two downstream uses: hypoglycemia early warning and insulin dosing decision support.<n>For early warning, we evaluate on real data from three clinical cohorts using event-level recall and false alarms per patient-day, metrics that reflect operational alarm burden rather than aggregate accuracy.<n>We show that models that look strong on real-data forecasting often fail to predict the direction, magnitude, or ranking of intervention effects, and choose poor insulin doses when evaluated under a clinically motivated cost.
• Score: 0.1104960878651584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clinical time-series forecasting is increasingly studied for decision support, yet standard aggregate metrics can obscure whether a model is actually useful for the task it is meant to serve. In safety-critical settings, low average error can coexist with dangerous failures in exactly the high-risk regimes that matter most. We present a task-aware evaluation framework for blood glucose forecasting built around two downstream uses: hypoglycemia early warning and insulin dosing decision support. For early warning, we evaluate on real data from three clinical cohorts using event-level recall and false alarms per patient-day, metrics that reflect operational alarm burden rather than aggregate accuracy. We show that models appearing acceptable overall, with recall above 0.9 on the full test set, can fail badly in the post-bolus slice, where insulin-on-board is elevated and missed warnings carry the greatest clinical consequences. Standard forecasting evaluation, however, does not test whether a model can reason about the effects of actions, a requirement for supporting insulin dosing decisions. We therefore add a second, interventional arm using the FDA-accepted UVA/Padova simulator, where we evaluate whether forecasters can predict glucose responses to altered insulin plans in paired factual/counterfactual scenarios. We show that models that look strong on real-data forecasting often fail to predict the direction, magnitude, or ranking of intervention effects, and choose poor insulin doses when evaluated under a clinically motivated cost. Taken together, the two arms reveal a consistent gap between forecasting accuracy and task-relevant usefulness. We release the benchmark, the standardized preprocessing pipeline for public cohorts, and the simulator-based interventional dataset as a reproducible toolkit.

Related papers

• CASCADE Conformal Prediction: Uncertainty-Adaptive Prediction Intervals for Two-Stage Clinical Decision Support [0.3681040154986213]
  We introduce CASCADE (Calibrated Adaptive Scaling via Conformal And Distributional Estimation), a novel conformal prediction framework.<n>By mapping Venn-Abers multi-probabilistic uncertainty directly to non-conformity scores, our framework achieves continuous risk adaptation.
  arXiv  Detail & Related papers  (2026-05-19T20:30:10Z)
• Uncertainty-aware Blood Glucose Prediction from Continuous Glucose Monitoring Data [0.0]
  We investigate uncertainty-aware neural network models for blood glucose prediction and adverse glycemic event identification in Type 1 diabetes.<n>We find that Transformer-based models equipped with evidential output heads provide the most effective uncertainty-aware framework.
  arXiv  Detail & Related papers  (2026-03-05T08:49:43Z)
• Towards Trustworthy Vital Sign Forecasting: Leveraging Uncertainty for Prediction Intervals [32.233133404873016]
  We present two methods for deriving PIs from the Reconstruction Uncertainty Estimate (RUE), an uncertainty measure well-suited to vital-sign forecasting.<n>We evaluate these methods on two large public datasets with minute- and hour-level sampling, representing high- and low-frequency health signals.
  arXiv  Detail & Related papers  (2025-09-01T10:03:26Z)
• SepsisLab: Early Sepsis Prediction with Uncertainty Quantification and Active Sensing [67.8991481023825]
  Sepsis is the leading cause of in-hospital mortality in the USA. Existing predictive models are usually trained on high-quality data with few missing information. For the potential high-risk patients with low confidence due to limited observations, we propose a robust active sensing algorithm.
  arXiv  Detail & Related papers  (2024-07-24T04:47:36Z)
• CenTime: Event-Conditional Modelling of Censoring in Survival Analysis [49.44664144472712]
  We introduce CenTime, a novel approach to survival analysis that directly estimates the time to event. Our method features an innovative event-conditional censoring mechanism that performs robustly even when uncensored data is scarce. Our results indicate that CenTime offers state-of-the-art performance in predicting time-to-death while maintaining comparable ranking performance.
  arXiv  Detail & Related papers  (2023-09-07T17:07:33Z)
• EventScore: An Automated Real-time Early Warning Score for Clinical Events [3.3039612529376625]
  We build an interpretable model for the early prediction of various adverse clinical events indicative of clinical deterioration. The model is evaluated on two datasets and four clinical events. Our model can be entirely automated without requiring any manually recorded features.
  arXiv  Detail & Related papers  (2021-02-11T11:55:08Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Prediction-Coherent LSTM-based Recurrent Neural Network for Safer Glucose Predictions in Diabetic People [4.692400531340393]
  We propose a LSTM-based recurrent neural network architecture and loss function that enhance the stability of predictions. The study is conducted on type 1 and type 2 diabetic people, with a focus on predictions made 30-minutes ahead of time.
  arXiv  Detail & Related papers  (2020-09-08T13:14:08Z)
• Enabling Counterfactual Survival Analysis with Balanced Representations [64.17342727357618]
  Survival data are frequently encountered across diverse medical applications, i.e., drug development, risk profiling, and clinical trials. We propose a theoretically grounded unified framework for counterfactual inference applicable to survival outcomes.
  arXiv  Detail & Related papers  (2020-06-14T01:15:00Z)
• Performance metrics for intervention-triggering prediction models do not reflect an expected reduction in outcomes from using the model [71.9860741092209]
  Clinical researchers often select among and evaluate risk prediction models. Standard metrics calculated from retrospective data are only related to model utility under certain assumptions. When predictions are delivered repeatedly throughout time, the relationship between standard metrics and utility is further complicated.
  arXiv  Detail & Related papers  (2020-06-02T16:26:49Z)
• Short Term Blood Glucose Prediction based on Continuous Glucose Monitoring Data [53.01543207478818]
  This study explores the use of Continuous Glucose Monitoring (CGM) data as input for digital decision support tools. We investigate how Recurrent Neural Networks (RNNs) can be used for Short Term Blood Glucose (STBG) prediction.
  arXiv  Detail & Related papers  (2020-02-06T16:39:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.