Scalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data

• URL: http://arxiv.org/abs/2606.01539v1
• Date: Mon, 01 Jun 2026 01:41:16 GMT
• Title: Scalable Counterfactual Risk Estimation for Rare Events in Longitudinal Data
• Abstract summary: We propose a principled subsampling and reweighting strategy for longitudinal survival data that can be applied to a range of existing causal effect estimators.<n>The proposed method substantially reduces computational burden while preserving consistency and improving estimation stability in rare-outcome settings.
• Score: 20.985792090921755
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating the causal effect of time-varying treatments on survival outcomes in large observational studies is computationally demanding, particularly when outcomes are rare. While g-formula-based methods such as the iterative conditional expectation (ICE) estimator provide a principled framework for longitudinal causal inference, they become computationally expensive, especially when bootstrap-based variance estimation is required. In addition, outcome rarity at each time point induces severe class imbalance, leading to instability and convergence issues in logistic regression and related models. To address these challenges, we propose a principled subsampling and reweighting strategy for longitudinal survival data that can be applied to a range of existing causal effect estimators in this setting, including the ICE estimator. The proposed method substantially reduces computational burden while preserving consistency and improving estimation stability in rare-outcome settings. We evaluate the method through simulations and validate it using a large-scale EHR cohort study on social and behavioral determinants of health (SBDH) and suicide risk, demonstrating its effectiveness for modeling rare outcomes in longitudinal data.

Related papers

• Surv-IPTB: An Attention-Based Model for Estimating Individual Probability of Treatment Benefit with Survival Data [0.6882042556551609]
  The proposed model, called Surv-IPTB, directly quantifies the probability that a specific patient will experience extended survival time under treatment versus control.<n>We reformulate IPTB estimation as a binary classification problem, leveraging pairwise patient comparisons across treatment and control cohorts.<n>An attention mechanism with learnable query-key transformations enables flexible, data-driven aggregation of pairwise comparisons.
  arXiv  Detail & Related papers  (2026-08-06T17:13:44Z)
• TraCeR: Transformer-Based Competing Risk Analysis with Longitudinal Covariates [0.0]
  TraCeR is a transformer-based survival analysis framework.<n>It estimates the hazard function from a sequence of measurements.<n>Experiments on multiple real-world datasets demonstrate substantial and statistically significant performance improvements.
  arXiv  Detail & Related papers  (2025-12-19T23:24:47Z)
• Learning Survival Models with Right-Censored Reporting Delays [10.312968200748118]
  Survival analysis is a statistical technique used to estimate the time until an event occurs.<n> adjusting for reporting delays under practical constraints remains a significant challenge in the insurance industry.<n>Our study addresses this challenge by jointly modeling the parametric hazard functions of event occurrences and report timings.
  arXiv  Detail & Related papers  (2025-10-06T01:16:57Z)
• Penalized Empirical Likelihood for Doubly Robust Causal Inference under Contamination in High Dimensions [0.720409153108429]
  We propose a doubly robust estimator for the average treatment effect in low sample size equations.<n>We show that the proposed confidence interval remain efficient compared to those competing estimates.
  arXiv  Detail & Related papers  (2025-07-23T11:58:54Z)
• A Generative Framework for Causal Estimation via Importance-Weighted Diffusion Distillation [55.53426007439564]
  Estimating individualized treatment effects from observational data is a central challenge in causal inference.<n>In inverse probability weighting (IPW) is a well-established solution to this problem, but its integration into modern deep learning frameworks remains limited.<n>We propose Importance-Weighted Diffusion Distillation (IWDD), a novel generative framework that combines the pretraining of diffusion models with importance-weighted score distillation.
  arXiv  Detail & Related papers  (2025-05-16T17:00:52Z)
• Self-Consistent Equation-guided Neural Networks for Censored Time-to-Event Data [11.550402345767141]
  We propose a novel approach to non-parametric estimation of the conditional survival functions using the generative adversarial networks leveraging self-consistent equations.<n>The proposed method is model-free and does not require any parametric assumptions on the structure of the conditional survival function.
  arXiv  Detail & Related papers  (2025-03-12T06:24:35Z)
• Evidential time-to-event prediction with calibrated uncertainty quantification [12.446406577462069]
  Time-to-event analysis provides insights into clinical prognosis and treatment recommendations.<n>We propose an evidential regression model specifically designed for time-to-event prediction.<n>We show that our model delivers both accurate and reliable performance, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-11-12T15:06:04Z)
• Risk and cross validation in ridge regression with correlated samples [72.59731158970894]
  We provide training examples for the in- and out-of-sample risks of ridge regression when the data points have arbitrary correlations.<n>We demonstrate that in this setting, the generalized cross validation estimator (GCV) fails to correctly predict the out-of-sample risk.<n>We further extend our analysis to the case where the test point has nontrivial correlations with the training set, a setting often encountered in time series forecasting.
  arXiv  Detail & Related papers  (2024-08-08T17:27:29Z)
• Predictability Analysis of Regression Problems via Conditional Entropy Estimations [1.8913544072080544]
  Conditional entropy estimators are developed to assess predictability in regression problems. Experiments on synthesized and real-world datasets demonstrate the robustness and utility of these estimators.
  arXiv  Detail & Related papers  (2024-06-06T07:59:19Z)
• Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations [84.42837346400151]
  Estimating counterfactual outcomes over time has the potential to unlock personalized healthcare. Existing causal inference approaches consider regular, discrete-time intervals between observations and treatment decisions. We propose a controllable simulation environment based on a model of tumor growth for a range of scenarios.
  arXiv  Detail & Related papers  (2022-06-16T17:15:15Z)
• SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data [83.50281440043241]
  We study the problem of inferring heterogeneous treatment effects from time-to-event data. We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
  arXiv  Detail & Related papers  (2021-10-26T20:13:17Z)
• Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score [59.75318183140857]
  Estimating causal effects from randomized experiments is central to clinical research. Most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control.
  arXiv  Detail & Related papers  (2020-12-17T21:10:10Z)
• Enabling Counterfactual Survival Analysis with Balanced Representations [64.17342727357618]
  Survival data are frequently encountered across diverse medical applications, i.e., drug development, risk profiling, and clinical trials. We propose a theoretically grounded unified framework for counterfactual inference applicable to survival outcomes.
  arXiv  Detail & Related papers  (2020-06-14T01:15:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.