KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing

• URL: http://arxiv.org/abs/2605.29524v1
• Date: Thu, 28 May 2026 07:40:24 GMT
• Title: KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing
• Abstract summary: KBF fingerprints model APIs using stable numerical recall near the knowledge boundary.<n>Findings: 7 of 27 platform model cells in a six-platform shadow API audit are statistically inconsistent with their reference endpoints, with inconsistencies concentrated on premium Claude endpoints.
• Score: 8.51962894711216
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Relay and reseller APIs increasingly intermediate access to large language models (LLMs), but users have no direct way to verify that a claimed endpoint is actually serving the advertised model. We introduce KBF, a low-cost black-box auditing protocol that fingerprints model APIs using stable numerical recall near the knowledge boundary. Across 16 production LLM endpoints, KBF flags all 155 economically relevant substitutions without rejecting any same-model controls, remains stable under deployment variation, detects high-separation mixed-routing attacks when only 5-10% of traffic is substituted, and finds that 7 of 27 platform model cells in a six-platform shadow API audit are statistically inconsistent with their reference endpoints, with inconsistencies concentrated on premium Claude endpoints.

Related papers

• HindsightBench: A Black-Box Behavioral Audit Protocol for Parametric Hindsight in Time-Indexed LLM Decision Tasks [1.416562653124415]
  We present HindsightBench, a behavioral audit protocol that profiles parametric hindsight at probe-level cost.<n>Applying it to 15 models from seven vendors on a 258-node vintage-correct macro panel yields three headline patterns.<n>We release the panel, frozen preregistrations, per-model audit rows with measured dollar costs, transcripts, and one-command regeneration.
  arXiv  Detail & Related papers  (2026-07-21T08:58:36Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models [52.93186090124315]
  We argue that token commitment can be learned as a reusable trace-state policy.<n>We introduce TraceLock, a lightweight plug-in controller that instantiates this policy for a frozen diffusion language model.
  arXiv  Detail & Related papers  (2026-05-23T18:23:46Z)
• Code-Centric Detection of Vulnerability-Fixing Commits: A Unified Benchmark and Empirical Study [4.512751676075442]
  We present a comprehensive evaluation of code language model based VFC detection through a unified framework.<n>We find no evidence that models acquire transferable security-relevant code understanding from code changes alone.<n>Group-stratified evaluation exposes approximately 17% performance drops compared to random splits.
  arXiv  Detail & Related papers  (2026-05-13T08:05:14Z)
• PLUME: Building a Network-Native Foundation Model for Wireless Traces via Protocol-Aware Tokenization [0.9558392439655014]
  Plume is a compact foundation model for 802.11 traces that learns from structured PDML dissections.<n>A protocol-aware tokenizer splits along the dissector field tree, emits gap tokens for timing, and normalizes identifiers, yielding 6.2x shorter sequences than BPE.
  arXiv  Detail & Related papers  (2026-03-13T23:14:43Z)
• Real Money, Fake Models: Deceptive Model Claims in Shadow APIs [26.860718016839126]
  Third-party services claim to provide access to official model services without regional limitations via indirect access.<n>Despite their widespread use, it remains unclear whether shadow APIs deliver outputs consistent with those of the official APIs.<n>These practices undermine the validity of scientific research, harm the interests of shadow API users, and damage the reputation of official model providers.
  arXiv  Detail & Related papers  (2026-03-02T14:33:05Z)
• IMMACULATE: A Practical LLM Auditing Framework via Verifiable Computation [49.796717294455796]
  We present IMMACULATE, a practical auditing framework that detects economically motivated deviations.<n>IMMACULATE selectively audits a small fraction of requests using verifiable computation, achieving strong detection guarantees while amortizing cryptographic overhead.
  arXiv  Detail & Related papers  (2026-02-26T07:21:02Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs [71.7892165868749]
  Commercial Large Language Model (LLM) APIs create a fundamental trust problem.<n>Users pay for specific models but have no guarantee that providers deliver them faithfully.<n>We formalize this model substitution problem and evaluate detection methods under realistic adversarial conditions.<n>We propose and evaluate the use of Trusted Execution Environments (TEEs) as one practical and robust solution.
  arXiv  Detail & Related papers  (2025-04-07T03:57:41Z)
• Do Large Language Model Benchmarks Test Reliability? [66.1783478365998]
  We investigate how well current benchmarks quantify model reliability.<n>Motivated by this gap in the evaluation of reliability, we propose the concept of so-called platinum benchmarks.<n>We evaluate a wide range of models on these platinum benchmarks and find that, indeed, frontier LLMs still exhibit failures on simple tasks.
  arXiv  Detail & Related papers  (2025-02-05T18:58:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.