Names Can Hurt: Spotting Slopsquatting Risks Caused by Package Name Hallucinations in Local Coding LLMs

• URL: http://arxiv.org/abs/2608.23897v1
• Date: Mon, 24 Aug 2026 23:10:49 GMT
• Title: Names Can Hurt: Spotting Slopsquatting Risks Caused by Package Name Hallucinations in Local Coding LLMs
• Abstract summary: When a code generating language model fabricates a Python package name, an adversary who has pre-registered that name on PyPI can convert that hallucination into a supply chain compromise.<n>We propose a two layer detector to counter this issue.<n>The detector is embedded in a LangGraph state machine that retries at escalating temperatures and, on repeated failure, routes to a stronger fallback model.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When a code generating language model fabricates a Python package name, an adversary who has pre-registered that name on PyPI can convert that hallucination into a supply chain compromise. This event has been termed as 'slopsquatting'. We propose a two layer detector to counter this issue. The first layer performs a deterministic PyPI existence check. The second is a Random Forest classifier trained on ten features derived from the package name and its PyPI metadata. An import name reconciler bridges the two, resolving cases such as 'import cv2' versus 'pip install opencv-python' without a security bypass. The detector is embedded in a LangGraph state machine that retries at escalating temperatures and, on repeated failure, routes to a stronger fallback model. Across 300 curated prompts, the pipeline produces hallucination free code on 76% of runs. The primary exhausts its retry budget on 28.7%; intra model retries recover roughly a quarter of those, and cross model fallback recovers a further 16.5% of the remainder. Four findings have been observed. First, half of the flagged hallucinations are packages already registered on PyPI, as low quality lookalikes of well known projects, caught by the classifier rather than the deterministic layer (e.g., pil, faiss, tabula, haystack). Second, hallucination rate scales almost linearly with prompt adversariality, from 0 to 10% on routine coding to 40 to 73% on slopsquat baits. Third, the weaker primary refused 6 of 10 direct baits unaided, suggesting recent instruction tuning provides a baseline defense. Fourth, when primary and fallback share a model family, approximately 84% of primary failures recur on the fallback, motivating cross family pairing. A user study (n = 24) reports mean satisfaction 4.4 out of 5 and 21 of 24 stated adoption intent.

Related papers

• Abliteration Is Not a Scalpel: Off-Target Effects of Refusal Removal on Decision Disposition Across Model Families [51.56484100374058]
  Abliteration - deleting a model's refusal direction from its weights - is the standard recipe behind popular "uncensored" open-weight models.<n>As a disposition probe we use 21,600 decisions under uncertainty, replayed through a frozen pipeline so the decision-layer model is the only variable.<n>Three effects replicate across both families (weeks-clustered CIs excluding zero)<n>A fourth effect reverses sign: the same operation makes Gemma-abliterated less confident and Qwen-abliterated more (family CIs non-overlapping)
  arXiv  Detail & Related papers  (2026-07-19T22:27:00Z)
• Bayesian-Calibrated Detection of Hallucinated Package Imports in AI-Assisted Code [0.0]
  calibrated layer exploits PyPI metadata signals to surface registered-but-suspicious packages that a binary registry detector misses.<n>We evaluate the calibration on a merged corpus of 1,734 snippets Python.
  arXiv  Detail & Related papers  (2026-06-11T21:15:39Z)
• The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort [51.56484100374058]
  Spracklen et al. showed that code-generating large language models hallucinate package names that do not exist on PyPI or npm.<n>We measure hallucination rates across 199,845 paired Python and JavaScript prompts validated against PyPI and npm master lists.<n>We identify a set of 127 package names (109 on PyPI, 18 on npm) that all five evaluated models invent identically.
  arXiv  Detail & Related papers  (2026-05-16T16:08:52Z)
• Delulu: A Verified Multi-Lingual Benchmark for Code Hallucination Detection in Fill-in-the-Middle Tasks [4.089259624354187]
  Large Language Models for code generation frequently produce hallucinations in Fillin-theMiddle (FIM) tasks.<n>We introduce Delulu, a verified multi-lingual benchmark of 1,951 FIM samples across 7 languages and 4 hallucination types.<n>We evaluate 11 openweight FIM models from five families spanning 0.5B-32B parameters.
  arXiv  Detail & Related papers  (2026-05-07T23:12:00Z)
• Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions [52.50730821321986]
  We present the first large-scale measurement study of version-level risk in large language models (LLMs)<n>We evaluate 10 LLMs on PinTrace, a curated benchmark of 1,000 Stack Overflow programming tasks.<n>Our findings reveal LLM version selection as a first-class, previously overlooked risk surface in LLM-based development.
  arXiv  Detail & Related papers  (2026-05-07T13:52:59Z)
• MEMRES: A Memory-Augmented Resolver with Confidence Cascade for Agentic Python Dependency Resolution [1.6376585841690154]
  MEMRES is an agentic system for Python dependency resolution.<n>On HG2.9K using Gemma-2 9B (10 GB VRAM) MEMRES resolves 2503 snippets of 2890 snippets.
  arXiv  Detail & Related papers  (2026-04-18T09:52:45Z)
• Assimilation Matters: Model-level Backdoor Detection in Vision-Language Pretrained Models [71.44858461725893]
  Given a model fine-tuned by an untrusted third party, determining whether the model has been injected with a backdoor is a critical and challenging problem.<n>Existing detection methods usually rely on prior knowledge of training dataset, backdoor triggers and targets.<n>We introduce Assimilation Matters in DETection (AMDET), a novel model-level detection framework that operates without any such prior knowledge.
  arXiv  Detail & Related papers  (2025-11-29T06:20:00Z)
• DySec: A Machine Learning-based Dynamic Analysis for Detecting Malicious Packages in PyPI Ecosystem [4.045165357831481]
  Malicious Python packages make software supply chains vulnerable by exploiting trust in open-source repositories like Python Package Index (PyPI)<n>Lack of real-time behavioral monitoring makes metadata inspection and static code analysis inadequate against advanced attack strategies.<n>We introduce DySec, a machine learning-based dynamic analysis framework for PyPI that uses eBPF kernel and user-level probes to monitor behaviors during package installation.
  arXiv  Detail & Related papers  (2025-03-01T03:20:42Z)
• PyPulse: A Python Library for Biosignal Imputation [58.35269251730328]
  We introduce PyPulse, a Python package for imputation of biosignals in both clinical and wearable sensor settings.<n>PyPulse's framework provides a modular and extendable framework with high ease-of-use for a broad userbase, including non-machine-learning bioresearchers.<n>We released PyPulse under the MIT License on Github and PyPI.
  arXiv  Detail & Related papers  (2024-12-09T11:00:55Z)
• Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference [72.61732440246954]
  Large pre-trained language models often lack logical consistency across test inputs. We propose a framework, ConCoRD, for boosting the consistency and accuracy of pre-trained NLP models. We show that ConCoRD consistently boosts accuracy and consistency of off-the-shelf closed-book QA and VQA models.
  arXiv  Detail & Related papers  (2022-11-21T21:58:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.