A Covariate-Adjusted Homogeneity Test with Application to Facial Recognition Accuracy Assessment

• URL: http://arxiv.org/abs/2307.08846v1
• Date: Mon, 17 Jul 2023 21:16:26 GMT
• Title: A Covariate-Adjusted Homogeneity Test with Application to Facial Recognition Accuracy Assessment
• Authors: Ngoc-Ty Nguyen, P. Jonathon Phillips, Larry Tang
• Abstract summary: Ordinal scores occur commonly in medical imaging studies and in black-box forensic studies. Our proposed test is applied to a face recognition study to identify statistically significant differences among five participant groups.
• Score: 0.3222802562733786
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Ordinal scores occur commonly in medical imaging studies and in black-box forensic studies \citep{Phillips:2018}. To assess the accuracy of raters in the studies, one needs to estimate the receiver operating characteristic (ROC) curve while accounting for covariates of raters. In this paper, we propose a covariate-adjusted homogeneity test to determine differences in accuracy among multiple rater groups. We derived the theoretical results of the proposed test and conducted extensive simulation studies to evaluate the finite sample performance of the proposed test. Our proposed test is applied to a face recognition study to identify statistically significant differences among five participant groups.

Related papers

• mlscorecheck: Testing the consistency of reported performance scores and experiments in machine learning [0.0]
  We have developed numerical techniques capable of identifying inconsistencies between reported performance scores and various experimental setups in machine learning problems. These consistency tests are integrated into the open-source package mlscorecheck.
  arXiv  Detail & Related papers  (2023-11-13T18:31:48Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Robustness Stress Testing in Medical Image Classification [26.094688963784254]
  We employ stress testing to assess model robustness and subgroup performance disparities in disease detection models. We apply stress tests to measure the robustness of disease detection models for chest X-ray and skin lesion images. Our experiments indicate that some models may yield more robust and equitable performance than others.
  arXiv  Detail & Related papers  (2023-08-14T02:02:56Z)
• A Double Machine Learning Approach to Combining Experimental and Observational Data [59.29868677652324]
  We propose a double machine learning approach to combine experimental and observational studies. Our framework tests for violations of external validity and ignorability under milder assumptions.
  arXiv  Detail & Related papers  (2023-07-04T02:53:11Z)
• Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation [54.72195809248172]
  We present a new estimator leveraging our proposed novel concept, that involves retrospective reshuffling of participants across experimental arms at the end of an RCT. We prove theoretically that such an estimator is more accurate than common estimators based on sample means.
  arXiv  Detail & Related papers  (2023-02-06T05:17:22Z)
• fAux: Testing Individual Fairness via Gradient Alignment [2.5329739965085785]
  We describe a new approach for testing individual fairness that does not have either requirement. We show that the proposed method effectively identifies discrimination on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-10-10T21:27:20Z)
• Statistical and Computational Phase Transitions in Group Testing [73.55361918807883]
  We study the group testing problem where the goal is to identify a set of k infected individuals carrying a rare disease. We consider two different simple random procedures for assigning individuals tests.
  arXiv  Detail & Related papers  (2022-06-15T16:38:50Z)
• Hierarchical Decision Ensembles- An inferential framework for uncertain Human-AI collaboration in forensic examinations [0.8122270502556371]
  We present an inferential framework for assessing the model and its output. The framework is designed to calibrate trust in forensic experts by bridging the gap between domain specific knowledge and predictive model results.
  arXiv  Detail & Related papers  (2021-10-31T08:07:43Z)
• Bayesian prognostic covariate adjustment [59.75318183140857]
  Historical data about disease outcomes can be integrated into the analysis of clinical trials in many ways. We build on existing literature that uses prognostic scores from a predictive model to increase the efficiency of treatment effect estimates.
  arXiv  Detail & Related papers  (2020-12-24T05:19:03Z)
• Cross-validation Confidence Intervals for Test Error [83.67415139421448]
  This work develops central limit theorems for crossvalidation and consistent estimators of its variance under weak stability conditions on the learning algorithm. Results are the first of their kind for the popular choice of leave-one-out cross-validation.
  arXiv  Detail & Related papers  (2020-07-24T17:40:06Z)
• Bayesian inference of infected patients in group testing with prevalence estimation [0.0]
  Group testing is a method of identifying infected patients by performing tests on a pool of specimens collected from patients. We show that the true-positive rate is improved by taking into account the credible interval of a point estimate of each patient.
  arXiv  Detail & Related papers  (2020-04-28T17:11:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.