Bayesian-Calibrated Detection of Hallucinated Package Imports in AI-Assisted Code

• URL: http://arxiv.org/abs/2606.13918v1
• Date: Thu, 11 Jun 2026 21:15:39 GMT
• Title: Bayesian-Calibrated Detection of Hallucinated Package Imports in AI-Assisted Code
• Abstract summary: calibrated layer exploits PyPI metadata signals to surface registered-but-suspicious packages that a binary registry detector misses.<n>We evaluate the calibration on a merged corpus of 1,734 snippets Python.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a Bayesian calibration layer for slopsquat detectors -- those that flag hallucinated package imports in code produced by large language models (LLMs). Where existing pipelines emit binary decisions (flag / do-not-flag), our layer emits a Beta-posterior probability per detection, derived from a 3-category epistemic taxonomy that explicitly classifies each prior as empirically calibrated, constructively argued, or engineering-judgement-traced. Beyond the primary 200/404 registry channel, the calibrated layer exploits PyPI metadata signals -- package age, release count, author descriptor, summary -- to surface registered-but-suspicious packages that a binary registry detector misses, which is the realistic post-LLM-emission attacker regime. The resulting risk-aware primitive is directly consumable by downstream CI gates and supports principled threshold decisions across detection rules. We evaluate the calibration on a merged corpus of 1,734 Python snippets -- a stratified 189-prompt BigCodeBench slice plus a 100-prompt niche-library stress-test set, generated across a six-model panel spanning four cloud models (Claude-Sonnet-4.6, Mistral-Large, DeepSeek-v4-pro, DeepSeek-R1) and two local open-weight code models (Mistral Codestral, Meta CodeLlama). Against a re-implemented binary baseline inspired by Mahmud et al. -- which shares its registry oracle with our ground truth and therefore serves as a degenerate upper bound rather than a genuine competitor -- the calibrated layer reproduces the strict-registry detections and introduces well-calibrated additional flags on the metadata channel. We assess detector asymmetry with a McNemar paired test and calibration with both a flagged-subset Expected Calibration Error and a strictly proper full-corpus Brier score.

Related papers

• Static Detection of Post-Quantum Cryptographic Algorithms in Stripped Binaries for Digital Forensic Examination and Migration Assurance [0.0]
  This paper presents Kestrel, a static analysis method that identifies an algorithm from the number-theoretic transform constant tables its arithmetic depends on.<n>It applies it to the standardised lattice schemes ML-KEM and ML-DSA, for which no prior method could confirm, from a shipped binary alone.
  arXiv  Detail & Related papers  (2026-08-25T20:17:08Z)
• Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit [8.665845754660458]
  AI-generated video detection GenVidBench and AIGVDBench are the de facto leaderboards.<n>Most evaluation protocols leave uncontrolled confounds that can inflate reported generalizations.<n>A 20-paper survey finds none applying all six standard controls that would catch this.<n>We combine them into an audited protocol and apply it to six representative feature sources.
  arXiv  Detail & Related papers  (2026-06-30T00:29:53Z)
• SrDetection: A Self-Referential Framework for Data Leakage Detection in Code Large Language Models [27.825867704129703]
  We introduce textbfself-textbfreferential leakage detection framework for both gray-box and black-box settings.<n>SrDetection generates semantically equivalent variants of a benchmark sample and detects leakage by contrasting the model's behavior on the original versus its variants.<n>Across different models and training stages, SrDetection improves average F1 by 21.52 points in the gray-box setting and 14.46 points in the black-box setting over strong baselines.
  arXiv  Detail & Related papers  (2026-06-29T05:48:42Z)
• From Attack Simulation to SIEM Rule: Deterministic Detection-as-Code Synthesis with Probe-Level Traceability [51.56484100374058]
  Security teams simulate attacks against their own systems to check whether their monitoring would catch a real intruder.<n>A human bridges that gap by hand, reading each finding and writing the corresponding Sigma rule.<n>We show this translation can be partially automated when probes are drawn from a locked corpus.
  arXiv  Detail & Related papers  (2026-06-03T14:26:25Z)
• VectorSmuggle: Steganographic Exfiltration in Embedding Stores and a Cryptographic Provenance Defense [0.0]
  Modern retrieval-augmented generation (RAG) systems convert sensitive content into high-dimensional embeddings and store them in vector databases that treat the resulting numerical artifacts as opaque.<n>We show this opens a class of steganographic exfiltration attacks.<n>An attacker with write access to the ingestion pipeline can hide payload data inside embeddings.
  arXiv  Detail & Related papers  (2026-05-13T16:44:20Z)
• Label-Free Detection of Governance Evidence Degradation in Risk Decision Systems [0.0]
  Risk decision systems in fraud detection and credit scoring operate under structural label absence.<n>No existing framework integrates drift detection with governance evidence assessment and operational response.<n>This paper presents a label-free governance monitoring extension to the Governance Drift Toolkit.
  arXiv  Detail & Related papers  (2026-04-20T05:46:15Z)
• Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection [105.14032334647932]
  Machine-generated texts (MGTs) pose risks such as disinformation and phishing, highlighting the need for reliable detection.<n> Metric-based methods, which extract statistically distinguishable features of MGTs, are often more practical than complex model-based methods that are prone to overfitting.<n>We propose a Markov-informed score calibration strategy that models two relationships of context detection scores that may aid calibration.
  arXiv  Detail & Related papers  (2026-02-08T16:06:12Z)
• Many Hands Make Light Work: An LLM-based Multi-Agent System for Detecting Malicious PyPI Packages [3.7667883869699597]
  Malicious code in open-source repositories such as PyPI poses a growing threat to software supply chains.<n>This paper presents LAMPS, a multi-agent system that employs collaborative language models to detect malicious PyPI packages.
  arXiv  Detail & Related papers  (2026-01-17T19:43:22Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)
• Semi-Supervised and Long-Tailed Object Detection with CascadeMatch [91.86787064083012]
  We propose a novel pseudo-labeling-based detector called CascadeMatch. Our detector features a cascade network architecture, which has multi-stage detection heads with progressive confidence thresholds. We show that CascadeMatch surpasses existing state-of-the-art semi-supervised approaches in handling long-tailed object detection.
  arXiv  Detail & Related papers  (2023-05-24T07:09:25Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement [53.69674636044927]
  We present EHSOD, an end-to-end hybrid-supervised object detection system. It can be trained in one shot on both fully and weakly-annotated data. It achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data.
  arXiv  Detail & Related papers  (2020-02-18T08:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.