NeurT-FDR: Controlling FDR by Incorporating Feature Hierarchy

• URL: http://arxiv.org/abs/2101.09809v1
• Date: Sun, 24 Jan 2021 21:55:10 GMT
• Title: NeurT-FDR: Controlling FDR by Incorporating Feature Hierarchy
• Authors: Lin Qiu, Nils Murrugarra-Llerena, V\'itor Silva, Lin Lin, Vernon M. Chinchilli
• Abstract summary: We propose NeurT-FDR which boosts statistical power and controls FDR for multiple hypothesis testing. We show that NeurT-FDR has strong FDR guarantees and makes substantially more discoveries in synthetic and real datasets.
• Score: 7.496622386458525
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Controlling false discovery rate (FDR) while leveraging the side information of multiple hypothesis testing is an emerging research topic in modern data science. Existing methods rely on the test-level covariates while ignoring possible hierarchy among the covariates. This strategy may not be optimal for complex large-scale problems, where hierarchical information often exists among those test-level covariates. We propose NeurT-FDR which boosts statistical power and controls FDR for multiple hypothesis testing while leveraging the hierarchy among test-level covariates. Our method parametrizes the test-level covariates as a neural network and adjusts the feature hierarchy through a regression framework, which enables flexible handling of high-dimensional features as well as efficient end-to-end optimization. We show that NeurT-FDR has strong FDR guarantees and makes substantially more discoveries in synthetic and real datasets compared to competitive baselines.

Related papers

• High-Dimensional False Discovery Rate Control for Dependent Variables [10.86851797584794]
  We propose a dependency-aware T-Rex selector that harnesses the dependency structure among variables. We prove that our variable penalization mechanism ensures FDR control. We formulate a fully integrated optimal calibration algorithm that concurrently determines the parameters of the graphical model and the T-Rex framework.
  arXiv  Detail & Related papers  (2024-01-28T22:56:16Z)
• REST: Enhancing Group Robustness in DNNs through Reweighted Sparse Training [49.581884130880944]
  Deep neural network (DNN) has been proven effective in various domains. However, they often struggle to perform well on certain minority groups during inference.
  arXiv  Detail & Related papers  (2023-12-05T16:27:54Z)
• DeepFDR: A Deep Learning-based False Discovery Rate Control Method for Neuroimaging Data [1.9207817188259122]
  Voxel-based multiple testing is widely used in neuroimaging data analysis. Traditional false discovery rate (FDR) control methods ignore the spatial dependence among the voxel-based tests. DeepFDR uses unsupervised deep learning-based image segmentation to address the voxel-based multiple testing problem.
  arXiv  Detail & Related papers  (2023-10-20T08:27:13Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Probabilistic Model Incorporating Auxiliary Covariates to Control FDR [6.270317798744481]
  Controlling False Discovery Rate (FDR) while leveraging the side information of multiple hypothesis testing is an emerging research topic in modern data science. We propose a deep Black-Box framework controlling FDR (named as NeurT-FDR) which boosts statistical power and controls FDR for multiple-hypothesis testing. We show that NeurT-FDR makes substantially more discoveries in three real datasets compared to competitive baselines.
  arXiv  Detail & Related papers  (2022-10-06T19:35:53Z)
• Two-stage Hypothesis Tests for Variable Interactions with FDR Control [10.750902543185802]
  We propose a two-stage testing procedure with false discovery rate (FDR) control, which is known as a less conservative multiple-testing correction. We demonstrate via comprehensive simulation studies that our two-stage procedure is more efficient than the classical BH procedure, with a comparable or improved statistical power.
  arXiv  Detail & Related papers  (2022-08-31T19:17:00Z)
• FEDNEST: Federated Bilevel, Minimax, and Compositional Optimization [53.78643974257301]
  Many contemporary ML problems fall under nested bilevel programming that subsumes minimax and compositional optimization. We propose FedNest: A federated alternating gradient method to address general nested problems.
  arXiv  Detail & Related papers  (2022-05-04T17:48:55Z)
• Latent Template Induction with Gumbel-CRFs [107.17408593510372]
  We explore the use of structured variational autoencoders to infer latent templates for sentence generation. As a structured inference network, we show that it learns interpretable templates during training.
  arXiv  Detail & Related papers  (2020-11-29T01:00:57Z)
• Lower bounds in multiple testing: A framework based on derandomized proxies [107.69746750639584]
  This paper introduces an analysis strategy based on derandomization, illustrated by applications to various concrete models. We provide numerical simulations of some of these lower bounds, and show a close relation to the actual performance of the Benjamini-Hochberg (BH) algorithm.
  arXiv  Detail & Related papers  (2020-05-07T19:59:51Z)
• Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees [49.91477656517431]
  Quantization-based solvers have been widely adopted in Federated Learning (FL) No existing methods enjoy all the aforementioned properties. We propose an intuitively-simple yet theoretically-simple method based on SIGNSGD to bridge the gap.
  arXiv  Detail & Related papers  (2020-02-25T15:12:15Z)
• Conditional Deep Gaussian Processes: multi-fidelity kernel learning [6.599344783327053]
  We propose the conditional DGP model in which the latent GPs are directly supported by the fixed lower fidelity data. Experiments with synthetic and high dimensional data show comparable performance against other multi-fidelity regression methods. We conclude that, with the low fidelity data and the hierarchical DGP structure, the effective kernel encodes the inductive bias for true function.
  arXiv  Detail & Related papers  (2020-02-07T14:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.