A Reproducibility Protocol for Cross-Implementation Evaluation of Post-Quantum ACVP Test Vectors

• URL: http://arxiv.org/abs/2608.13784v1
• Date: Thu, 13 Aug 2026 21:29:59 GMT
• Title: A Reproducibility Protocol for Cross-Implementation Evaluation of Post-Quantum ACVP Test Vectors
• Abstract summary: This study defines a product-neutral protocol for three public implementations of NIST ML-KEM.<n> Protocol v2 freezes provider-specific capabilities, applies one validation-error symmetrically, preserves every selected case, and separates bytes, validation verdicts, unsupported operations, and adapter errors.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Independent implementations of a cryptographic standard should reproduce the same known-answer results, yet agreement is meaningful only when the corpus, revisions, public interfaces, exclusions, and evidence are precisely stated. This study defines a product-neutral reproducibility protocol for three public implementations of NIST ML-KEM against a pinned public Automated Cryptographic Validation Protocol corpus. Protocol v2 freezes provider-specific capabilities, applies one validation-error taxonomy symmetrically, preserves every selected case, and separates bytes, validation verdicts, unsupported operations, and adapter errors. The source-built experiment evaluated @noble/post-quantum 0.7.0, liboqs 0.16.0, and Go 1.26.4. Across three repetitions, the required Cartesian product comprised 2,160 base records: all 1,650 declared executable evaluations matched the NIST oracle, and all 510 unsupported records matched Go's predeclared capability boundary. Pairwise agreement was complete on every executable overlap: 720 of 720 noble-liboqs records and 210 of 210 records for each Go pairing. A separate keyGen-ek-projection diagnostic matched all 150 Go encapsulation-key projections without counting them as full key generation. Three frozen controls independently exercised byte comparison, verdict comparison, and malformed-response error separation; each produced its exact predeclared outcome. No base failure, adapter error, or status instability occurred. The evidence establishes bounded author-run repeatability and exposes a practical standards gap: public ML-KEM packages provide materially different deterministic and validation-test surfaces. Independent external reproduction remains unobserved. The results do not establish certification, exhaustive correctness, side-channel resistance, secure integration, or production assurance.

Related papers

• Guardrailed Meta-Agent Loops: Stress-Testing Policy Pinning, Budget Bounds, and Crash Recovery [7.414694666903067]
  GuardrailLoop is a simulation-based testbed that makes three operational contracts jointly testable.<n>A hash-pinned policy fixes goals, scope, evaluation identity, budget, and release conditions.<n>It separates useful adaptation, state recovery, and repeated execution.
  arXiv  Detail & Related papers  (2026-09-10T21:22:08Z)
• VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement [57.86962208273888]
  We present an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations.<n>Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state.<n>We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time.
  arXiv  Detail & Related papers  (2026-09-02T20:36:45Z)
• BeTaL-GBI: Admission-Aware Benchmark Tuning and Full-Stack Verification of Geometric Belief Interfaces [2.894286977279531]
  This study examines whether an enterprise verification architecture can isolate interface failure, task competence, policy admissibility, and control integrity while keeping claims auditable.<n>BeTaL-GBI v0.2 applies Benchmark Tuning with an LLM-in-the-loop over 2,218,750,380 grid points, separating format admission from conditional performance.<n>Across 512 synthetic tasks, a 16-gate policy detects all 116 injected severe contradictions and accepts all 99 clean records. Hallucinator and evidence-forger surrogates are blocked with zero silent promotions.
  arXiv  Detail & Related papers  (2026-08-21T16:52:38Z)
• Beyond Local Accuracy: A Protocol-Level Identifiability Audit for Controlled LLM Reasoning Evaluation [6.047519836191728]
  We formalize a protocol-level identifiability audit over a finite behavioral policy class.<n>The audit requires zero model calls and resolves our diagnostic case.<n>This case shows how evaluation-design validity can be checked structurally before model inference.
  arXiv  Detail & Related papers  (2026-08-13T14:49:47Z)
• Specification-first convergence with an AI coding agent: a case study of dismantling a core architectural invariant across 189 files in a 717k-line codebase with no test oracle and no human code review [0.0]
  This paper reports a single, fully instrumented case study of a large-scale architectural dismantling by an AI coding agent.<n>Under the protocol described here, the agent completed it successfully.
  arXiv  Detail & Related papers  (2026-08-12T15:35:48Z)
• Does the way we write a theory change the program an LLM builds from it? A prospective randomized study of renderer format in LLM theory-to-program translation [0.0]
  A verbal theory does not run: translating it into an executable model requires choices about variables, interventions, and interactions.<n>We tested whether the presentation of otherwise identical theoretical content systematically changes the programs produced by theoretical language models.
  arXiv  Detail & Related papers  (2026-08-10T23:37:47Z)
• SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification [71.9783246097687]
  SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence.<n>For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation.<n>The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models.
  arXiv  Detail & Related papers  (2026-08-04T13:16:15Z)
• Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair [36.56438114281786]
  We study the gap between finding a correct patch and retaining, verifying, and submitting it.<n>We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation.
  arXiv  Detail & Related papers  (2026-07-27T16:05:23Z)
• Evidence-Grounded Ensemble Diagnosis of 802.11 Packet Captures: A Multi-Stage Pipeline with Deterministic Reliability Scoring [1.0170129555792935]
  Diagnosing 802.11 packet captures requires expert protocol knowledge, is slow, inconsistent across engineers, and unscalable.<n>LLM-based approaches sound plausible but fabricate protocol events absent from captures, produce uncalibrated confidence scores, and suffer evaluation bias when golden references are co-produced by the model under test.<n>We introduce PROBE, a multi-stage pipeline addressing all three failures.
  arXiv  Detail & Related papers  (2026-06-05T03:39:58Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• PBT-Bench: Benchmarking AI Agents on Property-Based Testing [29.035258104995204]
  We introduce PBT-Bench, a benchmark of 100 curated property-based testing problems across 40 real Python libraries.<n>Each problem injects one or more semantic bugs (365 in total, mean 3.65 per problem) designed so that default-strategy random inputs almost never trigger them.<n>Bug recall under the PBT-guided prompt ranges from 42.1% to 83.4% across models; under the open-ended baseline, from 31.4% to 76.7%.
  arXiv  Detail & Related papers  (2026-05-13T18:01:05Z)
• Beyond Code Reasoning: Specification-Anchored Auditing of Multi-Implementation Distributed Protocols [1.5229705287183657]
  SPECA is an audit framework that derives explicit, categorized security properties from natural-language specifications and reuses them across implementations.<n>On the RepoAudit benchmark, SPECA reaches 88.9% precision at 100% recall (F1=0.94) and surfaces 12 author-validated bugs beyond ground truth.<n>On the Sherlock Fusaka Audit Contest (10 targets, 366 submissions), SPECA recovers all 15 vulnerabilities expert-augmented and surfaces 4 fix-confirmed bugs.
  arXiv  Detail & Related papers  (2026-04-29T09:57:07Z)
• Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols [51.56484100374058]
  We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks.<n>For each instance, the interface records a baseline correctness bit and a post-step correctness bit.<n>These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines.
  arXiv  Detail & Related papers  (2026-04-20T13:25:40Z)
• SecCodeBench-V2 Technical Report [43.10947096543533]
  We introduce SecCodeBench-V2, a benchmark for evaluating Large Language Model (LLM) copilots' capabilities of generating secure code.<n>SecCodeBench-V2 comprises 98 generation and fix scenarios derived from Alibaba Group's industrial productions.<n>For each scenario, SecCodeBench-V2 provides executable proof-of-concept (PoC) test cases for both functional validation and security verification.
  arXiv  Detail & Related papers  (2026-02-17T10:47:06Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.