Finite Resources False Discovery Rate Control in Structured Hypothesis Spaces

• URL: http://arxiv.org/abs/2606.15393v1
• Date: Sat, 13 Jun 2026 16:56:52 GMT
• Title: Finite Resources False Discovery Rate Control in Structured Hypothesis Spaces
• Abstract summary: We present a framework for controlling the false discovery rate when each hypothesis is evidenced only by a finite count of null draws.<n>We present two decision rules that are both robust to structural mis-specification, yet offer a distinct trade-off between exact FDR control and statistical power.
• Score: 47.16593717985932
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific discovery relies on large-scale hypothesis testing. However, the capacity to identify true discoveries while controlling false discovery faces major challenges: obtaining relevant reference data (the null distribution) is resource-intensive, leaving finite-data uncertainty, and the procedure should account for the inherent structure in the hypothesis space, when such structure exists. Here, we present a framework for controlling the false discovery rate both when each hypothesis is evidenced only by a finite count of null draws, leaving its p-value uncertain, and when the hypothesis space carries arbitrary structure, requiring only that the structure be represented through a suitable reproducing kernel. We present two decision rules that are both robust to structural mis-specification, yet offer a distinct trade-off between exact FDR control and statistical power. The first rule guarantees exact FDR control; the second maximizes power by adapting mirror-statistic control into count space, utilizing an analytical framework to assess FDR control when exact mirror symmetry is relaxed. Furthermore, the tractability gained by the RKHS framework allows us to directly investigate finite-data uncertainties, which we leverage to suggest a policy for the efficient allocation of null distribution samples.

Related papers

• Finite Reliability Representations: Noise-Calibrated Belief-Space Covers for Reliable Decision-Making [0.0]
  We introduce Finite Reliability Representations (FRR), a framework for covering belief spaces by reliability cells.<n>FRR is formulated on beliefs rather than states and uses a cover rather than an equivalence quotient.<n>It applies to finite POMDPs, linear-Gaussian filters, locally linearized nonlinear filters, and particle-filter implementations.
  arXiv  Detail & Related papers  (2026-07-04T20:30:48Z)
• MMD-Balls as Credal Sets: A PAC-Bayesian Framework for Epistemic Uncertainty in Test-Time Adaptation [0.0]
  Test-time adaptation methods improve model performance under distribution shift.<n>Test-time adaptation methods lack formal guarantees connecting shift magnitude to prediction reliability.<n>We develop a PAC-Bayesian framework yielding bounds explicitly parameterized by the maximum mean discrepancy (MMD) between source and target distributions.
  arXiv  Detail & Related papers  (2026-05-20T22:22:20Z)
• Controlling False Discovery in Arbitrarily Structured Hypothesis Spaces via Reproducing Kernels [47.16593717985932]
  We introduce a framework that unifies continuous domains, graphs, and hierarchies under a single algorithm through kernel choice alone.<n>We provide two decision rules which we prove to control the False Discovery Rate (FDR)<n>We validate our method on two sources: spatial locations derived from high-dimensional real-world datasets, and a differential gene expression task utilizing protein-protein interaction graphs.
  arXiv  Detail & Related papers  (2026-05-17T17:42:56Z)
• The Digital Twin Counterfactual Framework: A Validation Architecture for Simulated Potential Outcomes [0.0]
  Digital Twin Counterfactual Framework (DTCF)<n>Five-level validation architecture converts unfalsifiable claim that simulator produces correct counterfactuals into falsifiable tests against observable data.<n> decomposition separates causal quantities into those that are marginally validated.<n>Joint causal claims become explicitly assumption-indexed.
  arXiv  Detail & Related papers  (2026-04-01T19:04:15Z)
• Observationally Informed Adaptive Causal Experimental Design [55.998153710215654]
  We propose Active Residual Learning, a new paradigm that leverages the observational model as a foundational prior.<n>This approach shifts the experimental focus from learning target causal quantities from scratch to efficiently estimating the residuals required to correct observational bias.<n> Experiments on synthetic and semi-synthetic benchmarks demonstrate that R-Design significantly outperforms baselines.
  arXiv  Detail & Related papers  (2026-03-04T06:52:37Z)
• LEC: Linear Expectation Constraints for False-Discovery Control in Selective Prediction and Routing Systems [95.35293543918762]
  Large language models (LLMs) often generate unreliable answers, while uncertainty methods fail to fully distinguish correct from incorrect predictions.<n>We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level.<n>We propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint.
  arXiv  Detail & Related papers  (2025-12-01T11:27:09Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• PAPRIKA: Private Online False Discovery Rate Control [27.698099204682105]
  We study False Discovery Rate (FDR) control in hypothesis testing under the constraint of differential privacy for the sample. We provide new private algorithms based on state-of-the-art results in non-private online FDR control.
  arXiv  Detail & Related papers  (2020-02-27T18:42:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.