Sequential Audit Sampling with Statistical Guarantees

• URL: http://arxiv.org/abs/2604.06116v1
• Date: Tue, 07 Apr 2026 17:26:23 GMT
• Title: Sequential Audit Sampling with Statistical Guarantees
• Abstract summary: This study formulates audit sampling with additional, sequentially collected items as a sequential testing problem for a finite population under sampling without replacement.<n>We define null and alternative hypotheses in terms of a deviation rate, specify stopping and decision rules, and formulate exact sequential boundary conditions in terms of finite-population error probabilities.<n>The exact design yields ex ante control of decision error probabilities, and the simulation-based implementation approximates that design while allowing the computation of expected stopping times.
• Score: 6.101839518775968
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Financial statement auditing is conducted under a risk-based evidence approach to obtain reasonable assurance. In practice, auditors often perform additional sampling or related procedures when an initial sample does not provide a sufficient basis for a conclusion. Across jurisdictions, current standards and practice manuals acknowledge such extensions, while the statistical design of sequential audit procedures has not been fully explored. This study formulates audit sampling with additional, sequentially collected items as a sequential testing problem for a finite population under sampling without replacement. We define null and alternative hypotheses in terms of a tolerable deviation rate, specify stopping and decision rules, and formulate exact sequential boundary conditions in terms of finite-population error probabilities. For practical implementation, we calibrate those boundaries by Monte Carlo simulation at least-favorable deviation rates. The exact design yields ex ante control of decision error probabilities, and the simulation-based implementation approximates that design while allowing the computation of expected stopping times. The framework is most naturally suited to attribute auditing and deviation-rate auditing, especially tests of controls, and it can be extended to one-sided, two-stage, and truncated designs.

Related papers

• Local Violation Certification for Linear Predict-Then-Optimize Pipelines [0.0]
  We present a framework for local violation certification designed specifically for linear decision pipelines under input uncertainty.<n>We mathematically demonstrate that standard sampling methods fail efficiently for rare violations, motivating a direct structural approach.<n>We demonstrate our approach on an economic power dispatch system subject to emissions regulations, delivering precise, auditable risk assessments.
  arXiv  Detail & Related papers  (2026-08-05T05:58:06Z)
• Sequential Fairness Auditing with Limited Output Access [6.933020649563105]
  External evaluations are becoming increasingly central to the governance of AI systems.<n>In practice, independent auditors often have limited access to deployed models and must rely on query-based interactions.<n>Most existing fairness evaluation methods assume static datasets and fixed-sample statistical tests.<n>We formulate fairness auditing as a tolerance-aware sequential hypothesis-testing problem under limited model output access.
  arXiv  Detail & Related papers  (2026-06-29T14:17:33Z)
• Adaptive auditing of AI systems with anytime-valid guarantees [8.752768302067638]
  We introduce a hypothesis testing framework from two 'dueling' perspectives.<n>We formalize the auditor as conducting 'testing by betting', which translates into simultaneous e-processes for testing the dueling null hypotheses.<n>Our proposed testing procedures maintain anytime-valid type-I error control, outperform pre-specified testing methods, and can reach statistically rigorous conclusions sometimes with as few as 20 observations.
  arXiv  Detail & Related papers  (2026-05-07T22:33:48Z)
• Bridging Conformal Prediction and Scenario Optimization: Discarded Constraints and Modular Risk Allocation [0.0]
  This paper revisits that connection directly from a systems-and-control viewpoint.<n>We introduce a simple modular composition rule that combines several blockwise calibration certificates into a single joint guarantee.
  arXiv  Detail & Related papers  (2026-03-19T18:37:58Z)
• Data Fusion with Distributional Equivalence Test-then-pool [5.5945020319921435]
  Existing test-then-pool (TTP) procedures address this concern by testing for equality of control outcomes between historical and concurrent trials.<n>We develop a new TTP framework that fuses control arms while rigorously controlling the Type-I error rate of the final treatment effect test.
  arXiv  Detail & Related papers  (2026-03-12T12:38:35Z)
• Predicting fixed-sample test decisions enables anytime-valid inference [0.3222802562733787]
  We introduce a simple procedure that transforms any fixed-sample hypothesis test into an anytime-valid test.<n>We ensure Type-I error control and near-optimal power with substantial sample savings when the null hypothesis is false.<n>In areas such as clinical trials, stopping early and safely can ensure that subjects receive the best treatments and accelerate the development of effective therapies.
  arXiv  Detail & Related papers  (2026-02-14T20:17:51Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering [55.15192437680943]
  Generative models lack rigorous statistical guarantees for their outputs.<n>We propose a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee.<n>This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example.
  arXiv  Detail & Related papers  (2024-10-02T15:26:52Z)
• Few-Shot Scenario Testing for Autonomous Vehicles Based on Neighborhood Coverage and Similarity [8.97909097472183]
  Testing and evaluating safety performance of autonomous vehicles (AVs) is essential before the large-scale deployment. The number of testing scenarios permissible for a specific AV is severely limited by tight constraints on testing budgets and time. We formulate this problem for the first time the "few-shot testing" (FST) problem and propose a systematic framework to address this challenge.
  arXiv  Detail & Related papers  (2024-02-02T04:47:14Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations [44.71254888821376]
  We provide the first type-I-error and expected-rejection-time guarantees under general non-data generating processes. We show how to apply our results to inference on parameters defined by estimating equations, such as average treatment effects.
  arXiv  Detail & Related papers  (2022-12-29T18:37:08Z)
• Sequential Kernelized Independence Testing [77.237958592189]
  We design sequential kernelized independence tests inspired by kernelized dependence measures.<n>We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations [67.40641255908443]
  We identify limitations of model-randomization-based sanity checks for the purpose of evaluating explanations. Top-down model randomization preserves scales of forward pass activations with high probability.
  arXiv  Detail & Related papers  (2022-11-22T18:52:38Z)
• Stopping Criterion Design for Recursive Bayesian Classification: Analysis and Decision Geometry [11.399206131178104]
  We propose a geometric interpretation over the state posterior progression. We show that confidence thresholds defined over maximum of the state posteriors suffer from stiffness. We then propose a new stopping/termination criterion with a geometrical insight to overcome the limitations.
  arXiv  Detail & Related papers  (2020-07-30T16:21:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.