A Cautionary Tale on Integrating Studies with Disparate Outcome Measures for Causal Inference

• URL: http://arxiv.org/abs/2505.11014v1
• Date: Fri, 16 May 2025 09:08:28 GMT
• Title: A Cautionary Tale on Integrating Studies with Disparate Outcome Measures for Causal Inference
• Authors: Harsh Parikh, Trang Quynh Nguyen, Elizabeth A. Stuart, Kara E. Rudolph, Caleb H. Miles,
• Abstract summary: This paper studies whether and when integrating studies with disparate outcome measures leads to efficiency gains.<n>We introduce three sets of assumptions -- with varying degrees of strength -- linking both outcome measures.<n>Our findings emphasize the need for careful assumption selection when fusing datasets with differing outcome measures.
• Score: 5.330251011543498
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Data integration approaches are increasingly used to enhance the efficiency and generalizability of studies. However, a key limitation of these methods is the assumption that outcome measures are identical across datasets -- an assumption that often does not hold in practice. Consider the following opioid use disorder (OUD) studies: the XBOT trial and the POAT study, both evaluating the effect of medications for OUD on withdrawal symptom severity (not the primary outcome of either trial). While XBOT measures withdrawal severity using the subjective opiate withdrawal scale, POAT uses the clinical opiate withdrawal scale. We analyze this realistic yet challenging setting where outcome measures differ across studies and where neither study records both types of outcomes. Our paper studies whether and when integrating studies with disparate outcome measures leads to efficiency gains. We introduce three sets of assumptions -- with varying degrees of strength -- linking both outcome measures. Our theoretical and empirical results highlight a cautionary tale: integration can improve asymptotic efficiency only under the strongest assumption linking the outcomes. However, misspecification of this assumption leads to bias. In contrast, a milder assumption may yield finite-sample efficiency gains, yet these benefits diminish as sample size increases. We illustrate these trade-offs via a case study integrating the XBOT and POAT datasets to estimate the comparative effect of two medications for opioid use disorder on withdrawal symptoms. By systematically varying the assumptions linking the SOW and COW scales, we show potential efficiency gains and the risks of bias. Our findings emphasize the need for careful assumption selection when fusing datasets with differing outcome measures, offering guidance for researchers navigating this common challenge in modern data integration.

Related papers

• Learning Causally Predictable Outcomes from Psychiatric Longitudinal Data [6.09170287691728]
  Causal inference in longitudinal biomedical data remains a central challenge.<n>Our algorithm learns non-negative, clinically interpretable weights for outcome aggregation.<n>Our algorithm consistently outperforms state-of-the-art methods in recovering causal effects.
  arXiv  Detail & Related papers  (2025-06-19T21:56:30Z)
• Path-specific effects for pulse-oximetry guided decisions in critical care [22.98557361265164]
  This study causally investigates how racial discrepancies in oximetry measurements affect invasive ventilation in ICU settings.<n>We employ a causal inference-based approach using path-specific effects to isolate the impact of bias by race on clinical decision-making.
  arXiv  Detail & Related papers  (2025-06-14T06:45:53Z)
• Data Fusion for Partial Identification of Causal Effects [62.56890808004615]
  We propose a novel partial identification framework that enables researchers to answer key questions.<n>Is the causal effect positive or negative? and How severe must assumption violations be to overturn this conclusion?<n>We apply our framework to the Project STAR study, which investigates the effect of classroom size on students' third-grade standardized test performance.
  arXiv  Detail & Related papers  (2025-05-30T07:13:01Z)
• Statistical Learning for Heterogeneous Treatment Effects: Pretraining, Prognosis, and Prediction [40.96453902709292]
  We propose pretraining strategies that leverage a phenomenon in real-world applications.<n>In medicine, components of the same biological signaling pathways frequently influence both baseline risk and treatment response.<n>We use this structure to incorporate "side information" and develop models that can exploit synergies between risk prediction and causal effect estimation.
  arXiv  Detail & Related papers  (2025-05-01T05:12:14Z)
• Estimating Heterogeneous Treatment Effects on Survival Outcomes Using Counterfactual Censoring Unbiased Transformations [1.9785304593748243]
  Methods for estimating heterogeneous treatment effects (HTE) from observational data have largely focused on continuous or binary outcomes. We develop censoring unbiased transformations (CUTs) for survival outcomes both with and without competing risks.
  arXiv  Detail & Related papers  (2024-01-20T16:17:06Z)
• The Blessings of Multiple Treatments and Outcomes in Treatment Effect Estimation [53.81860494566915]
  Existing studies leveraged proxy variables or multiple treatments to adjust for confounding bias. In many real-world scenarios, there is greater interest in studying the effects on multiple outcomes. We show that parallel studies of multiple outcomes involved in this setting can assist each other in causal identification.
  arXiv  Detail & Related papers  (2023-09-29T14:33:48Z)
• Treatment Effect Risk: Bounds and Inference [58.442274475425144]
  Since the average treatment effect measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE distribution. Some bounds can also be interpreted as summarizing a complex CATE function into a single metric and are of interest independently of being a bound.
  arXiv  Detail & Related papers  (2022-01-15T17:21:26Z)
• SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data [83.50281440043241]
  We study the problem of inferring heterogeneous treatment effects from time-to-event data. We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
  arXiv  Detail & Related papers  (2021-10-26T20:13:17Z)
• Enabling Counterfactual Survival Analysis with Balanced Representations [64.17342727357618]
  Survival data are frequently encountered across diverse medical applications, i.e., drug development, risk profiling, and clinical trials. We propose a theoretically grounded unified framework for counterfactual inference applicable to survival outcomes.
  arXiv  Detail & Related papers  (2020-06-14T01:15:00Z)
• Learning for Dose Allocation in Adaptive Clinical Trials with Safety Constraints [84.09488581365484]
  Phase I dose-finding trials are increasingly challenging as the relationship between efficacy and toxicity of new compounds becomes more complex. Most commonly used methods in practice focus on identifying a Maximum Tolerated Dose (MTD) by learning only from toxicity events. We present a novel adaptive clinical trial methodology that aims at maximizing the cumulative efficacies while satisfying the toxicity safety constraint with high probability.
  arXiv  Detail & Related papers  (2020-06-09T03:06:45Z)
• Overly Optimistic Prediction Results on Imbalanced Data: a Case Study of Flaws and Benefits when Applying Over-sampling [13.463035357173045]
  We focus on one specific type of methodological flaw: applying over-sampling before partitioning the data into mutually exclusive training and testing sets. We show how this causes the results to be biased using two artificial datasets and reproduce results of studies in which this flaw was identified.
  arXiv  Detail & Related papers  (2020-01-15T12:53:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.