Structure-Adaptive Sequential Testing for Online False Discovery Rate Control

• URL: http://arxiv.org/abs/2003.00113v1
• Date: Fri, 28 Feb 2020 23:16:44 GMT
• Title: Structure-Adaptive Sequential Testing for Online False Discovery Rate Control
• Authors: Bowen Gang, Wenguang Sun, and Weinan Wang
• Abstract summary: This work develops a new class of structure--adaptive sequential testing (SAST) rules for online false discover rate (FDR) control. A key element in our proposal is a new alpha--investment algorithm that precisely characterizes the gains and losses in sequential decision making.
• Score: 1.456699007803424
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Consider the online testing of a stream of hypotheses where a real--time decision must be made before the next data point arrives. The error rate is required to be controlled at {all} decision points. Conventional \emph{simultaneous testing rules} are no longer applicable due to the more stringent error constraints and absence of future data. Moreover, the online decision--making process may come to a halt when the total error budget, or alpha--wealth, is exhausted. This work develops a new class of structure--adaptive sequential testing (SAST) rules for online false discover rate (FDR) control. A key element in our proposal is a new alpha--investment algorithm that precisely characterizes the gains and losses in sequential decision making. SAST captures time varying structures of the data stream, learns the optimal threshold adaptively in an ongoing manner and optimizes the alpha-wealth allocation across different time periods. We present theory and numerical results to show that the proposed method is valid for online FDR control and achieves substantial power gain over existing online testing rules.

Related papers

• Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time Adaptation [49.53202761595912]
  Continual Test-Time Adaptation involves adapting a pre-trained source model to continually changing unsupervised target domains. We analyze the challenges of this task: online environment, unsupervised nature, and the risks of error accumulation and catastrophic forgetting. We propose an uncertainty-aware buffering approach to identify and aggregate significant samples with high certainty from the unsupervised, single-pass data stream.
  arXiv  Detail & Related papers  (2024-07-12T15:48:40Z)
• CAP: A General Algorithm for Online Selective Conformal Prediction with FCR Control [4.137346786534721]
  It is important to control the real-time false coverage-statement rate (FCR) which measures the overall miscoverage level. We develop a general framework named CAP that performs an adaptive pick rule on historical data to construct a calibration set. We prove that CAP can achieve an exact selection-conditional coverage guarantee in the finite-sample and distribution-free regimes.
  arXiv  Detail & Related papers  (2024-03-12T15:07:20Z)
• Online multiple testing with e-values [37.0397290998274]
  A scientist wishes to make as many discoveries as possible while ensuring the number of false discoveries is controlled. Prior methods for FDR control in the online setting have focused on formulating algorithms when specific dependency structures are assumed to exist between the test statistics of each hypothesis. Our algorithm, e-LOND, provides FDR control under arbitrary, possibly unknown, dependence.
  arXiv  Detail & Related papers  (2023-11-10T22:14:47Z)
• State Sequences Prediction via Fourier Transform for Representation Learning [111.82376793413746]
  We propose State Sequences Prediction via Fourier Transform (SPF), a novel method for learning expressive representations efficiently. We theoretically analyze the existence of structural information in state sequences, which is closely related to policy performance and signal regularity. Experiments demonstrate that the proposed method outperforms several state-of-the-art algorithms in terms of both sample efficiency and performance.
  arXiv  Detail & Related papers  (2023-10-24T14:47:02Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• Sequential Kernelized Independence Testing [101.22966794822084]
  We design sequential kernelized independence tests inspired by kernelized dependence measures. We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
  We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-03-09T01:55:59Z)
• Learning Robust Output Control Barrier Functions from Safe Expert Demonstrations [50.37808220291108]
  This paper addresses learning safe output feedback control laws from partial observations of expert demonstrations. We first propose robust output control barrier functions (ROCBFs) as a means to guarantee safety. We then formulate an optimization problem to learn ROCBFs from expert demonstrations that exhibit safe system behavior.
  arXiv  Detail & Related papers  (2021-11-18T23:21:00Z)
• Online Control of the False Discovery Rate under "Decision Deadlines" [1.4213973379473654]
  Online testing procedures aim to control the extent of false discoveries over a sequence of hypothesis tests. Our method controls the false discovery rate (FDR) at every stage of testing, as well as at adaptively chosen stopping times.
  arXiv  Detail & Related papers  (2021-10-04T17:28:09Z)
• Non-Episodic Learning for Online LQR of Unknown Linear Gaussian System [0.0]
  We propose an online non-episodic algorithm that gains knowledge about the system from a single trajectory. We characterize the almost sure convergence rates of identification and control, and reveal an optimal trade-off between exploration and exploitation.
  arXiv  Detail & Related papers  (2021-03-24T15:51:28Z)
• Dynamic Algorithms for Online Multiple Testing [38.45810475976042]
  We derive new algorithms for online multiple testing that provably control false discovery exceedance (FDX) We demonstrate that our algorithms achieve higher power in a variety of synthetic experiments. SupLORD is the first non-trivial algorithm, to our knowledge, that can control FDR at stopping times in the online setting.
  arXiv  Detail & Related papers  (2020-10-26T23:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.