SAFFRON and LORD Ensure Online Control of the False Discovery Rate Under Positive Dependence

• URL: http://arxiv.org/abs/2110.08161v1
• Date: Fri, 15 Oct 2021 15:43:24 GMT
• Title: SAFFRON and LORD Ensure Online Control of the False Discovery Rate Under Positive Dependence
• Authors: Aaron Fisher
• Abstract summary: Some of the most popular online methods include alpha investing, LORD++ (hereafter, LORD), and SAFFRON. These three methods have been shown to provide online control of the "modified" false discovery rate (mFDR) Our work bolsters these results by showing that SAFFRON and LORD additionally ensure online control of the FDR under nonnegative dependence.
• Score: 1.4213973379473654
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online testing procedures assume that hypotheses are observed in sequence, and allow the significance thresholds for upcoming tests to depend on the test statistics observed so far. Some of the most popular online methods include alpha investing, LORD++ (hereafter, LORD), and SAFFRON. These three methods have been shown to provide online control of the "modified" false discovery rate (mFDR). However, to our knowledge, they have only been shown to control the traditional false discovery rate (FDR) under an independence condition on the test statistics. Our work bolsters these results by showing that SAFFRON and LORD additionally ensure online control of the FDR under nonnegative dependence. Because alpha investing can be recovered as a special case of the SAFFRON framework, the same result applies to this method as well. Our result also allows for certain forms of adaptive stopping times, for example, stopping after a certain number of rejections have been observed.

Related papers

• Ranking by Lifts: A Cost-Benefit Approach to Large-Scale A/B Tests [0.0]
  A/B testers conducting large-scale tests want to be able to control false rejections of the null. This work develops a decision-theoretic framework for maximizing profits subject to false discovery rate (FDR) control.
  arXiv  Detail & Related papers  (2024-07-01T07:40:08Z)
• Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking [48.68044413117397]
  PINOSE trains a probing model on offline self-consistency checking results, thereby circumventing the need for human-annotated data. It examines various aspects of internal states prior to response decoding, contributing to more effective detection of factual inaccuracies.
  arXiv  Detail & Related papers  (2024-04-10T05:00:35Z)
• Model-free Test Time Adaptation for Out-Of-Distribution Detection [62.49795078366206]
  We propose a Non-Parametric Test Time textbfAdaptation framework for textbfDistribution textbfDetection (abbr) abbr utilizes online test samples for model adaptation during testing, enhancing adaptability to changing data distributions. We demonstrate the effectiveness of abbr through comprehensive experiments on multiple OOD detection benchmarks.
  arXiv  Detail & Related papers  (2023-11-28T02:00:47Z)
• A powerful rank-based correction to multiple testing under positive dependency [48.098218835606055]
  We develop a novel multiple hypothesis testing correction with family-wise error rate (FWER) control. Our proposed algorithm $textttmax-rank$ is conceptually straight-forward, relying on the use of a $max$-operator in the rank domain of computed test statistics.
  arXiv  Detail & Related papers  (2023-11-17T22:44:22Z)
• Online multiple testing with e-values [37.0397290998274]
  A scientist wishes to make as many discoveries as possible while ensuring the number of false discoveries is controlled. Prior methods for FDR control in the online setting have focused on formulating algorithms when specific dependency structures are assumed to exist between the test statistics of each hypothesis. Our algorithm, e-LOND, provides FDR control under arbitrary, possibly unknown, dependence.
  arXiv  Detail & Related papers  (2023-11-10T22:14:47Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• Sequential Kernelized Independence Testing [101.22966794822084]
  We design sequential kernelized independence tests inspired by kernelized dependence measures. We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Online Control of the False Discovery Rate under "Decision Deadlines" [1.4213973379473654]
  Online testing procedures aim to control the extent of false discoveries over a sequence of hypothesis tests. Our method controls the false discovery rate (FDR) at every stage of testing, as well as at adaptively chosen stopping times.
  arXiv  Detail & Related papers  (2021-10-04T17:28:09Z)
• Tracking disease outbreaks from sparse data with Bayesian inference [55.82986443159948]
  The COVID-19 pandemic provides new motivation for estimating the empirical rate of transmission during an outbreak. Standard methods struggle to accommodate the partial observability and sparse data common at finer scales. We propose a Bayesian framework which accommodates partial observability in a principled manner.
  arXiv  Detail & Related papers  (2020-09-12T20:37:33Z)
• Structure-Adaptive Sequential Testing for Online False Discovery Rate Control [1.456699007803424]
  This work develops a new class of structure--adaptive sequential testing (SAST) rules for online false discover rate (FDR) control. A key element in our proposal is a new alpha--investment algorithm that precisely characterizes the gains and losses in sequential decision making.
  arXiv  Detail & Related papers  (2020-02-28T23:16:44Z)
• PAPRIKA: Private Online False Discovery Rate Control [27.698099204682105]
  We study False Discovery Rate (FDR) control in hypothesis testing under the constraint of differential privacy for the sample. We provide new private algorithms based on state-of-the-art results in non-private online FDR control.
  arXiv  Detail & Related papers  (2020-02-27T18:42:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.