Conformal novelty detection with false discovery rate control at the boundary

• URL: http://arxiv.org/abs/2601.02610v1
• Date: Tue, 06 Jan 2026 00:02:03 GMT
• Title: Conformal novelty detection with false discovery rate control at the boundary
• Authors: Zijun Gao, Etienne Roquain, Daniel Xiang,
• Abstract summary: Conformal novelty detection is a classical machine learning task.<n>Recent work has shown that the BH procedure applied to conformal p-values controls the false discovery rate (FDR)
• Score: 3.10490198369453
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Conformal novelty detection is a classical machine learning task for which uncertainty quantification is essential for providing reliable results. Recent work has shown that the BH procedure applied to conformal p-values controls the false discovery rate (FDR). Unfortunately, the BH procedure can lead to over-optimistic assessments near the rejection threshold, with an increase of false discoveries at the margin as pointed out by Soloff et al. (2024). This issue is solved therein by the support line (SL) correction, which is proven to control the boundary false discovery rate (bFDR) in the independent, non-conformal setting. The present work extends the SL method to the conformal setting: first, we show that the SL procedure can violate the bFDR control in this specific setting. Second, we propose several alternatives that provably control the bFDR in the conformal setting. Finally, numerical experiments with both synthetic and real data support our theoretical findings and show the relevance of the new proposed procedures.

Related papers

• Learning False Discovery Rate Control via Model-Based Neural Networks [14.45679797184966]
  We introduce a learning-augmented enhancement of the T-Rex Selector framework that narrows the gap between the realized false discovery proportion (FDP) and the target false discovery rate (FDR)<n>Our approach replaces the analytical FDP estimator with a neural network trained solely on diverse synthetic datasets, enabling a substantially tighter and more accurate approximation of the FDP.
  arXiv  Detail & Related papers  (2026-02-05T15:53:11Z)
• LEC: Linear Expectation Constraints for False-Discovery Control in Selective Prediction and Routing Systems [95.35293543918762]
  Large language models (LLMs) often generate unreliable answers, while uncertainty methods fail to fully distinguish correct from incorrect predictions.<n>We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level.<n>We propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint.
  arXiv  Detail & Related papers  (2025-12-01T11:27:09Z)
• Discovering Causal Relationships using Proxy Variables under Unmeasured Confounding [42.70985072862832]
  Inferring causal relationships between variable pairs in the observational study is crucial but challenging.<n>We develop a general nonparametric approach that accommodates both discrete and continuous settings for testing causal hypothesis under unmeasured confounders.<n>We demonstrate the effectiveness of our approach through extensive simulations and real-world data from the Intensive Care Data and World Values Survey.
  arXiv  Detail & Related papers  (2025-10-20T05:13:12Z)
• DP-SPRT: Differentially Private Sequential Probability Ratio Tests [18.783606628556342]
  We revisit Wald's celebrated Sequential Probability Ratio Test for sequential tests of two simple hypotheses, under privacy constraints.<n>We propose DP-SPRT, a wrapper that can be calibrated to achieve desired error probabilities and privacy constraints.
  arXiv  Detail & Related papers  (2025-08-08T15:09:13Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• On the Adversarial Robustness of Benjamini Hochberg [0.06144680854063938]
  The Benjamini-Hochberg (BH) procedure is widely used to control the false detection rate (FDR) in multiple testing.<n>Considering this control could be relied upon in critical safety/security contexts, we investigate its adversarial robustness.
  arXiv  Detail & Related papers  (2025-01-06T21:41:53Z)
• Adaptive novelty detection with false discovery rate guarantee [1.8249324194382757]
  We propose a flexible method to control the false discovery rate (FDR) on detected novelties in finite samples. Inspired by the multiple testing literature, we propose variants of AdaDetect that are adaptive to the proportion of nulls. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.
  arXiv  Detail & Related papers  (2022-08-13T17:14:55Z)
• Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
  We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-03-09T01:55:59Z)
• Mitigating the Mutual Error Amplification for Semi-Supervised Object Detection [92.52505195585925]
  We propose a Cross Teaching (CT) method, aiming to mitigate the mutual error amplification by introducing a rectification mechanism of pseudo labels. In contrast to existing mutual teaching methods that directly treat predictions from other detectors as pseudo labels, we propose the Label Rectification Module (LRM)
  arXiv  Detail & Related papers  (2022-01-26T03:34:57Z)
• On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times [51.61278695776151]
  Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively. Recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks. Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks.
  arXiv  Detail & Related papers  (2021-01-11T19:43:12Z)
• Lower bounds in multiple testing: A framework based on derandomized proxies [107.69746750639584]
  This paper introduces an analysis strategy based on derandomization, illustrated by applications to various concrete models. We provide numerical simulations of some of these lower bounds, and show a close relation to the actual performance of the Benjamini-Hochberg (BH) algorithm.
  arXiv  Detail & Related papers  (2020-05-07T19:59:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.