Refining CART Models for Covariate Shift with Importance Weight

• URL: http://arxiv.org/abs/2410.20978v1
• Date: Mon, 28 Oct 2024 12:53:23 GMT
• Title: Refining CART Models for Covariate Shift with Importance Weight
• Authors: Mingyang Cai, Thomas Klausch, Mark A. van de Wiel,
• Abstract summary: This paper introduces an adaptation of Classification and Regression Trees (CART) that incorporates importance weighting to address these distributional differences effectively. We evaluate the effectiveness of this method through simulation studies and apply it to real-world medical data, showing significant improvements in predictive accuracy.
• Score: 0.0
• License:
• Abstract: Machine learning models often face challenges in medical applications due to covariate shifts, where discrepancies between training and target data distributions can decrease predictive accuracy. This paper introduces an adaptation of Classification and Regression Trees (CART) that incorporates importance weighting to address these distributional differences effectively. By assigning greater weight to training samples that closely represent the target distribution, our approach modifies the CART model to improve performance in the presence of covariate shift. We evaluate the effectiveness of this method through simulation studies and apply it to real-world medical data, showing significant improvements in predictive accuracy. The results indicate that this weighted CART approach can be valuable in medical and other fields where covariate shift poses challenges, enabling more reliable predictions across diverse data distributions.

Related papers

• Optimizing importance weighting in the presence of sub-population shifts [0.0]
  A distribution shift between the training and test data can severely harm performance of machine learning models. We argue that existing weightings for determining the weights are suboptimal, as they neglect the increase of the variance of the estimated model due to the finite sample size of the training data. We propose a bi-level optimization procedure in which the weights and model parameters are optimized simultaneously.
  arXiv  Detail & Related papers  (2024-10-18T09:21:10Z)
• Understanding Calibration of Deep Neural Networks for Medical Image Classification [3.461503547789351]
  This study explores model performance and calibration under different training regimes. We consider fully supervised training, as well as rotation-based self-supervised method with and without transfer learning. Our study reveals that factors such as weight distributions and the similarity of learned representations correlate with the calibration trends observed in the models.
  arXiv  Detail & Related papers  (2023-09-22T18:36:07Z)
• Characterizing Out-of-Distribution Error via Optimal Transport [15.284665509194134]
  Methods of predicting a model's performance on OOD data without labels are important for machine learning safety. We introduce a novel method for estimating model performance by leveraging optimal transport theory. We show that our approaches significantly outperform existing state-of-the-art methods with an up to 3x lower prediction error.
  arXiv  Detail & Related papers  (2023-05-25T01:37:13Z)
• Density-Aware Personalized Training for Risk Prediction in Imbalanced Medical Data [89.79617468457393]
  Training models with imbalance rate (class density discrepancy) may lead to suboptimal prediction. We propose a framework for training models for this imbalance issue. We demonstrate our model's improved performance in real-world medical datasets.
  arXiv  Detail & Related papers  (2022-07-23T00:39:53Z)
• Undersmoothing Causal Estimators with Generative Trees [0.0]
  Inferring individualised treatment effects from observational data can unlock the potential for targeted interventions. It is, however, hard to infer these effects from observational data. In this paper, we explore a novel generative tree based approach that tackles model misspecification directly.
  arXiv  Detail & Related papers  (2022-03-16T11:59:38Z)
• FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributes [51.02407217197623]
  We propose a two-stage training algorithm named FAIRIF. It minimizes the loss over the reweighted data set where the sample weights are computed. We show that FAIRIF yields models with better fairness-utility trade-offs against various types of bias.
  arXiv  Detail & Related papers  (2022-01-15T05:14:48Z)
• Causal Effect Variational Autoencoder with Uniform Treatment [50.895390968371665]
  Causal effect variational autoencoder (CEVAE) are trained to predict the outcome given observational treatment data. Uniform treatment variational autoencoders (UTVAE) are trained with uniform treatment distribution using importance sampling.
  arXiv  Detail & Related papers  (2021-11-16T17:40:57Z)
• Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization [89.73665256847858]
  We show that out-of-distribution performance is strongly correlated with in-distribution performance for a wide range of models and distribution shifts. Specifically, we demonstrate strong correlations between in-distribution and out-of-distribution performance on variants of CIFAR-10 & ImageNet. We also investigate cases where the correlation is weaker, for instance some synthetic distribution shifts from CIFAR-10-C and the tissue classification dataset Camelyon17-WILDS.
  arXiv  Detail & Related papers  (2021-07-09T19:48:23Z)
• Predicting with Confidence on Unseen Distributions [90.68414180153897]
  We connect domain adaptation and predictive uncertainty literature to predict model accuracy on challenging unseen distributions. We find that the difference of confidences (DoC) of a classifier's predictions successfully estimates the classifier's performance change over a variety of shifts. We specifically investigate the distinction between synthetic and natural distribution shifts and observe that despite its simplicity DoC consistently outperforms other quantifications of distributional difference.
  arXiv  Detail & Related papers  (2021-07-07T15:50:18Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.