Causal Covariate Shift Correction using Fisher information penalty

• URL: http://arxiv.org/abs/2502.15756v1
• Date: Tue, 11 Feb 2025 15:51:59 GMT
• Title: Causal Covariate Shift Correction using Fisher information penalty
• Authors: Behraj Khan, Behroz Mirza, Tahir Syed,
• Abstract summary: This work takes a distributed density estimation angle to the training setting where data are temporally distributed.<n>The penalty improves accuracy by $12.9%$ over the full-dataset baseline.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Evolving feature densities across batches of training data bias cross-validation, making model selection and assessment unreliable (\cite{sugiyama2012machine}). This work takes a distributed density estimation angle to the training setting where data are temporally distributed. \textit{Causal Covariate Shift Correction ($C^{3}$)}, accumulates knowledge about the data density of a training batch using Fisher Information, and using it to penalize the loss in all subsequent batches. The penalty improves accuracy by $12.9\%$ over the full-dataset baseline, by $20.3\%$ accuracy at maximum in batchwise and $5.9\%$ at minimum in foldwise benchmarks.

Related papers

• Efficient Learning Under Density Shift in Incremental Settings Using Cramér-Rao-Based Regularization [0.0]
  This work takes a distributed density estimation angle to the problem where data are temporally distributed.<n>It processes data in batches and allows a neural network to treat a batch as training data.<n>$C2A$ achieves $19%$ accuracy at maximum against state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-02-18T16:00:10Z)
• Mitigating covariate shift in non-colocated data with learned parameter priors [0.0]
  We present textitFragmentation-induced co-shift remediation ($FIcsR$), which minimizes an $f$-divergence between a fragment's covariate distribution and that of the standard cross-validation baseline. We run extensive classification experiments on multiple data classes, over $40$ datasets, and with data batched over multiple sequence lengths. The results are promising under all these conditions; with improved accuracy against batch and fold state-of-the-art by more than $5%$ and $10%$, respectively.
  arXiv  Detail & Related papers  (2024-11-10T15:48:29Z)
• Machine Learning Force Fields with Data Cost Aware Training [94.78998399180519]
  Machine learning force fields (MLFF) have been proposed to accelerate molecular dynamics (MD) simulation. Even for the most data-efficient MLFFs, reaching chemical accuracy can require hundreds of frames of force and energy labels. We propose a multi-stage computational framework -- ASTEROID, which lowers the data cost of MLFFs by leveraging a combination of cheap inaccurate data and expensive accurate data.
  arXiv  Detail & Related papers  (2023-06-05T04:34:54Z)
• Transfer Knowledge from Head to Tail: Uncertainty Calibration under Long-tailed Distribution [24.734851889816206]
  Current calibration techniques treat different classes equally and implicitly assume that the distribution of training data is balanced. We propose a novel knowledge-transferring-based calibration method by estimating the importance weights for samples of tail classes.
  arXiv  Detail & Related papers  (2023-04-13T13:48:18Z)
• Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent [97.64313409741614]
  We propose to enforce a emphconsistency property which states that predictions of the model on its own generated data are consistent across time. We show that our novel training objective yields state-of-the-art results for conditional and unconditional generation in CIFAR-10 and baseline improvements in AFHQ and FFHQ.
  arXiv  Detail & Related papers  (2023-02-17T18:45:04Z)
• Adapting to Continuous Covariate Shift via Online Density Ratio Estimation [64.8027122329609]
  Dealing with distribution shifts is one of the central challenges for modern machine learning. We propose an online method that can appropriately reuse historical information. Our density ratio estimation method is proven to perform well by enjoying a dynamic regret bound.
  arXiv  Detail & Related papers  (2023-02-06T04:03:33Z)
• Sample-dependent Adaptive Temperature Scaling for Improved Calibration [95.7477042886242]
  Post-hoc approach to compensate for neural networks being wrong is to perform temperature scaling. We propose to predict a different temperature value for each input, allowing us to adjust the mismatch between confidence and accuracy. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets.
  arXiv  Detail & Related papers  (2022-07-13T14:13:49Z)
• Test-time Batch Statistics Calibration for Covariate Shift [66.7044675981449]
  We propose to adapt the deep models to the novel environment during inference. We present a general formulation $alpha$-BN to calibrate the batch statistics. We also present a novel loss function to form a unified test time adaptation framework Core.
  arXiv  Detail & Related papers  (2021-10-06T08:45:03Z)
• On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks [94.23884467360521]
  We show that state-of-the art models trained on QQP and WikiQA each have only $2.4%$ average precision when evaluated on realistically imbalanced test data. By creating balanced training data with more informative negative examples, active learning greatly improves average precision to $32.5%$ on QQP and $20.1%$ on WikiQA.
  arXiv  Detail & Related papers  (2020-10-10T21:56:27Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.