PAPRIKA: Private Online False Discovery Rate Control

• URL: http://arxiv.org/abs/2002.12321v2
• Date: Wed, 21 Oct 2020 03:06:54 GMT
• Title: PAPRIKA: Private Online False Discovery Rate Control
• Authors: Wanrong Zhang, Gautam Kamath, Rachel Cummings
• Abstract summary: We study False Discovery Rate (FDR) control in hypothesis testing under the constraint of differential privacy for the sample. We provide new private algorithms based on state-of-the-art results in non-private online FDR control.
• Score: 27.698099204682105
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In hypothesis testing, a false discovery occurs when a hypothesis is incorrectly rejected due to noise in the sample. When adaptively testing multiple hypotheses, the probability of a false discovery increases as more tests are performed. Thus the problem of False Discovery Rate (FDR) control is to find a procedure for testing multiple hypotheses that accounts for this effect in determining the set of hypotheses to reject. The goal is to minimize the number (or fraction) of false discoveries, while maintaining a high true positive rate (i.e., correct discoveries). In this work, we study False Discovery Rate (FDR) control in multiple hypothesis testing under the constraint of differential privacy for the sample. Unlike previous work in this direction, we focus on the online setting, meaning that a decision about each hypothesis must be made immediately after the test is performed, rather than waiting for the output of all tests as in the offline setting. We provide new private algorithms based on state-of-the-art results in non-private online FDR control. Our algorithms have strong provable guarantees for privacy and statistical performance as measured by FDR and power. We also provide experimental results to demonstrate the efficacy of our algorithms in a variety of data environments.

Related papers

• PSBD: Prediction Shift Uncertainty Unlocks Backdoor Detection [57.571451139201855]
  Prediction Shift Backdoor Detection (PSBD) is a novel method for identifying backdoor samples in deep neural networks. PSBD is motivated by an intriguing Prediction Shift (PS) phenomenon, where poisoned models' predictions on clean data often shift away from true labels towards certain other labels. PSBD identifies backdoor training samples by computing the Prediction Shift Uncertainty (PSU), the variance in probability values when dropout layers are toggled on and off during model inference.
  arXiv  Detail & Related papers  (2024-06-09T15:31:00Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [55.17761802332469]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications. We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Federated Causal Discovery from Heterogeneous Data [70.31070224690399]
  We propose a novel FCD method attempting to accommodate arbitrary causal models and heterogeneous data. These approaches involve constructing summary statistics as a proxy of the raw data to protect data privacy. We conduct extensive experiments on synthetic and real datasets to show the efficacy of our method.
  arXiv  Detail & Related papers  (2024-02-20T18:53:53Z)
• Online multiple testing with e-values [37.0397290998274]
  A scientist wishes to make as many discoveries as possible while ensuring the number of false discoveries is controlled. Prior methods for FDR control in the online setting have focused on formulating algorithms when specific dependency structures are assumed to exist between the test statistics of each hypothesis. Our algorithm, e-LOND, provides FDR control under arbitrary, possibly unknown, dependence.
  arXiv  Detail & Related papers  (2023-11-10T22:14:47Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• Sequential Kernelized Independence Testing [101.22966794822084]
  We design sequential kernelized independence tests inspired by kernelized dependence measures. We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Model-Free Sequential Testing for Conditional Independence via Testing by Betting [8.293345261434943]
  The proposed test allows researchers to analyze an incoming i.i.d. data stream with any arbitrary dependency structure. We allow the processing of data points online as soon as they arrive and stop data acquisition once significant results are detected.
  arXiv  Detail & Related papers  (2022-10-01T20:05:33Z)
• Private Sequential Hypothesis Testing for Statisticians: Privacy, Error Rates, and Sample Size [24.149533870085175]
  We study the sequential hypothesis testing problem under a slight variant of differential privacy, known as Renyi differential privacy. We present a new private algorithm based on Wald's Sequential Probability Ratio Test (SPRT) that also gives strong theoretical privacy guarantees.
  arXiv  Detail & Related papers  (2022-04-10T04:15:50Z)
• SAFFRON and LORD Ensure Online Control of the False Discovery Rate Under Positive Dependence [1.4213973379473654]
  Some of the most popular online methods include alpha investing, LORD++ (hereafter, LORD), and SAFFRON. These three methods have been shown to provide online control of the "modified" false discovery rate (mFDR) Our work bolsters these results by showing that SAFFRON and LORD additionally ensure online control of the FDR under nonnegative dependence.
  arXiv  Detail & Related papers  (2021-10-15T15:43:24Z)
• Online Control of the False Discovery Rate under "Decision Deadlines" [1.4213973379473654]
  Online testing procedures aim to control the extent of false discoveries over a sequence of hypothesis tests. Our method controls the false discovery rate (FDR) at every stage of testing, as well as at adaptively chosen stopping times.
  arXiv  Detail & Related papers  (2021-10-04T17:28:09Z)
• Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design [63.48989885374238]
  When the infection prevalence of a disease is low, Dorfman showed 80 years ago that testing groups of people can prove more efficient than testing people individually. Our goal in this paper is to propose new group testing algorithms that can operate in a noisy setting.
  arXiv  Detail & Related papers  (2020-04-26T23:41:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.