Statistical Inference under Performativity

• URL: http://arxiv.org/abs/2505.18493v2
• Date: Thu, 19 Jun 2025 01:25:32 GMT
• Title: Statistical Inference under Performativity
• Authors: Xiang Li, Yunai Li, Huiying Zhong, Lihua Lei, Zhun Deng,
• Abstract summary: We build a central limit theorem for estimation and inference under performativity.<n>We investigate prediction-powered inference (PPI) under performativity based on a small labeled dataset and a much larger dataset of machine-learning predictions.<n>To the best of our knowledge, this paper is the first one to establish statistical inference under performativity.
• Score: 12.935979571180464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Performativity of predictions refers to the phenomena that prediction-informed decisions may influence the target they aim to predict, which is widely observed in policy-making in social sciences and economics. In this paper, we initiate the study of statistical inference under performativity. Our contribution is two-fold. First, we build a central limit theorem for estimation and inference under performativity, which enables inferential purposes in policy-making such as constructing confidence intervals or testing hypotheses. Second, we further leverage the derived central limit theorem to investigate prediction-powered inference (PPI) under performativity, which is based on a small labeled dataset and a much larger dataset of machine-learning predictions. This enables us to obtain more precise estimation and improved confidence regions for the model parameter (i.e., policy) of interest in performative prediction. We demonstrate the power of our framework by numerical experiments. To the best of our knowledge, this paper is the first one to establish statistical inference under performativity, which brings up new challenges and inference settings that we believe will add significant values to policy-making, statistics, and machine learning.

Related papers

• Another look at inference after prediction [0.0]
  Prediction-based (PB) inference has emerged to accommodate statistical analysis using a large volume of predictions together with a small amount of gold-standard data.<n>The goals of PB inference are (i) to mitigate bias from errors in predictions and (ii) to improve efficiency relative to traditional inference using only the gold-standard data.<n>We show that, with a simple modification, Prediction-powered Inference (PPI) can be adjusted to provide theoretically justified improvements in efficiency.
  arXiv  Detail & Related papers  (2024-11-29T18:12:50Z)
• An Information Theoretic Perspective on Conformal Prediction [15.194199235970242]
  Conformal Prediction (CP) constructs prediction sets guaranteed to contain the true answer with a user-specified probability.<n>In this work, we leverage information theory to connect conformal prediction to other notions of uncertainty.
  arXiv  Detail & Related papers  (2024-05-03T14:43:07Z)
• Selective Temporal Knowledge Graph Reasoning [70.11788354442218]
  Temporal Knowledge Graph (TKG) aims to predict future facts based on given historical ones. Existing TKG reasoning models are unable to abstain from predictions they are uncertain. We propose an abstention mechanism for TKG reasoning, which helps the existing models make selective, instead of indiscriminate, predictions.
  arXiv  Detail & Related papers  (2024-04-02T06:56:21Z)
• Prediction De-Correlated Inference: A safe approach for post-prediction inference [0.8192907805418583]
  Inference in this setting is often called post-prediction inference. We propose a novel assumption-lean framework for statistical inference under post-prediction setting, called Prediction De-Correlated Inference (PDC) Our approach is safe, in the sense that PDC can automatically adapt to any black-box machine-learning model and consistently outperform the supervised counterparts.
  arXiv  Detail & Related papers  (2023-12-11T16:04:48Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Prediction-Powered Inference [68.97619568620709]
  Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning.
  arXiv  Detail & Related papers  (2023-01-23T18:59:28Z)
• Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions [59.284907093349425]
  Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. We provide a language for describing how training data influences predictions, through a causal framework. Our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone.
  arXiv  Detail & Related papers  (2022-07-28T17:36:24Z)
• Double Robust Representation Learning for Counterfactual Prediction [68.78210173955001]
  We propose a novel scalable method to learn double-robust representations for counterfactual predictions. We make robust and efficient counterfactual predictions for both individual and average treatment effects. The algorithm shows competitive performance with the state-of-the-art on real world and synthetic data.
  arXiv  Detail & Related papers  (2020-10-15T16:39:26Z)
• Robust Validation: Confident Predictions Even When Distributions Shift [19.327409270934474]
  We describe procedures for robust predictive inference, where a model provides uncertainty estimates on its predictions rather than point predictions. We present a method that produces prediction sets (almost exactly) giving the right coverage level for any test distribution in an $f$-divergence ball around the training population. An essential component of our methodology is to estimate the amount of expected future data shift and build robustness to it.
  arXiv  Detail & Related papers  (2020-08-10T17:09:16Z)
• Counterfactual Predictions under Runtime Confounding [74.90756694584839]
  We study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data. We propose a doubly-robust procedure for learning counterfactual prediction models in this setting.
  arXiv  Detail & Related papers  (2020-06-30T15:49:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.