LLM in the Shell: Generative Honeypots

• URL: http://arxiv.org/abs/2309.00155v3
• Date: Mon, 23 Sep 2024 12:51:00 GMT
• Title: LLM in the Shell: Generative Honeypots
• Abstract summary: Honeypots are essential tools in cybersecurity for early detection, threat intelligence gathering, and analysis of attacker's behavior. This work introduces shelLM, a dynamic and realistic software honeypot based on Large Language Models that generates Linux-like shell output.
• Score: 0.26249027950824505
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Honeypots are essential tools in cybersecurity for early detection, threat intelligence gathering, and analysis of attacker's behavior. However, most of them lack the required realism to engage and fool human attackers long-term. Being easy to distinguish honeypots strongly hinders their effectiveness. This can happen because they are too deterministic, lack adaptability, or lack deepness. This work introduces shelLM, a dynamic and realistic software honeypot based on Large Language Models that generates Linux-like shell output. We designed and implemented shelLM using cloud-based LLMs. We evaluated if shelLM can generate output as expected from a real Linux shell. The evaluation was done by asking cybersecurity researchers to use the honeypot and give feedback if each answer from the honeypot was the expected one from a Linux shell. Results indicate that shelLM can create credible and dynamic answers capable of addressing the limitations of current honeypots. ShelLM reached a TNR of 0.90, convincing humans it was consistent with a real Linux shell. The source code and prompts for replicating the experiments have been publicly available.

Related papers

• AdvancedShelLM: A Stateful Multi-Agent LLM Honeypot for SSH Deception [0.3262230127283452]
  We present a new LLM-based honeypot design that uses a multi-agent, multi-LLM architecture to address the limitations of the previous shelLM LLM honeypot.<n>Our honeypot, called AdvancedShelLM, uses two LLM agents, a Manager and a Worker, that better understand the commands while reducing incorrect responses and increasing deception.
  arXiv  Detail & Related papers  (2026-06-26T11:39:05Z)
• Honeyval: A Comprehensive Evaluation Framework for LLM-powered HTTP Honeypots [22.377298744054954]
  Honeypots are decoy systems mimicking real system components designed to defend against cyber attacks.<n>Recently, LLMs increasingly serve as simulation backbones for honeypots.<n>Honeyval is a comprehensive evaluation framework for LLM-powered HTTP honeypots.
  arXiv  Detail & Related papers  (2026-05-28T14:02:27Z)
• Can LLMs Threaten Human Survival? Benchmarking Potential Existential Threats from LLMs via Prefix Completion [25.01615836807345]
  We study whether large language models (LLMs) produce unpredictable outputs that could pose substantive threats to human safety.<n>We propose textscExistBench, a benchmark designed to evaluate such risks.<n>Experiments on 10 LLMs reveal that LLM-generated content indicates existential threats.
  arXiv  Detail & Related papers  (2025-11-24T14:34:13Z)
• VelLMes: A high-interaction AI-based deception framework [0.3499870393443268]
  This paper presents an AI-based deception framework called VelLMes.<n>VelLMes is designed to be attacked by humans, so interactivity and realism are key for its performance.<n>We evaluate the generative capabilities and the deception capabilities.
  arXiv  Detail & Related papers  (2025-10-08T13:00:23Z)
• LLMHoney: A Real-Time SSH Honeypot with Large Language Model-Driven Dynamic Response Generation [0.0]
  This Report presents LLMHoney, an SSH honeypot that leverages Large Language Models (LLMs) to generate realistic, dynamic command outputs in real time.<n>LLMs integrate a dictionary-based virtual file system to handle common commands with low latency.<n>We report comprehensive metrics including accuracy (exact-match, Cosine Similarity, Jaro-Winkler Similarity, Levenshtein Similarity and BLEU score), response latency and memory overhead.
  arXiv  Detail & Related papers  (2025-09-01T13:28:00Z)
• Aligning LLMs to Be Robust Against Prompt Injection [55.07562650579068]
  We show that alignment can be a powerful tool to make LLMs more robust against prompt injection attacks. Our method -- SecAlign -- first builds an alignment dataset by simulating prompt injection attacks. Our experiments show that SecAlign robustifies the LLM substantially with a negligible hurt on model utility.
  arXiv  Detail & Related papers  (2024-10-07T19:34:35Z)
• Output Scouting: Auditing Large Language Models for Catastrophic Responses [1.5703117863274307]
  Recent high profile incidents in which the use of Large Language Models (LLMs) resulted in significant harm to individuals have brought about a growing interest in AI safety. One reason LLM safety issues occur is that models often have at least some non-zero probability of producing harmful outputs. We propose output scouting: an approach that aims to generate semantically fluent outputs to a given prompt matching any target probability distribution.
  arXiv  Detail & Related papers  (2024-10-04T18:18:53Z)
• Kov: Transferable and Naturalistic Black-Box LLM Attacks using Markov Decision Processes and Tree Search [1.223779595809275]
  Eliciting harmful behavior from large language models (LLMs) is an important task to ensure the proper alignment and safety of the models. This work frames the red-teaming problem as a Markov decision process (MDP) and uses Monte Carlo tree search to find harmful behaviors of black-box, closed-source LLMs. The proposed algorithm, Kov, trains on white-box LLMs to optimize the adversarial attacks and periodically evaluates responses from the black-box LLM to guide the search towards more harmful black-box behaviors.
  arXiv  Detail & Related papers  (2024-08-11T20:31:52Z)
• Get my drift? Catching LLM Task Drift with Activation Deltas [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users. We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set. We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• Assessing LLMs in Malicious Code Deobfuscation of Real-world Malware Campaigns [7.776434991976473]
  This paper studies the deobfuscation capabilities of large language models (LLMs) We evaluate four LLMs with real-world malicious scripts used in the notorious Emotet malware campaign. Our results indicate that while not absolutely accurate yet, some LLMs can efficiently deobfuscate such payloads.
  arXiv  Detail & Related papers  (2024-04-30T17:06:27Z)
• CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models [6.931433424951554]
  Large language models (LLMs) introduce new security risks, but there are few comprehensive evaluation suites to measure and reduce these risks. We present BenchmarkName, a novel benchmark to quantify LLM security risks and capabilities. We evaluate multiple state-of-the-art (SOTA) LLMs, including GPT-4, Mistral, Meta Llama 3 70B-Instruct, and Code Llama.
  arXiv  Detail & Related papers  (2024-04-19T20:11:12Z)
• Coercing LLMs to do and reveal (almost) anything [80.8601180293558]
  It has been shown that adversarial attacks on large language models (LLMs) can "jailbreak" the model into making harmful statements. We argue that the spectrum of adversarial attacks on LLMs is much larger than merely jailbreaking.
  arXiv  Detail & Related papers  (2024-02-21T18:59:13Z)
• Feedback Loops With Language Models Drive In-Context Reward Hacking [78.9830398771605]
  We show that feedback loops can cause in-context reward hacking (ICRH) We identify and study two processes that lead to ICRH: output-refinement and policy-refinement. As AI development accelerates, the effects of feedback loops will proliferate.
  arXiv  Detail & Related papers  (2024-02-09T18:59:29Z)
• Assured LLM-Based Software Engineering [51.003878077888686]
  This paper is an outline of the content of the keynote by Mark Harman at the International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering, Monday 15th April 2024, Lisbon, Portugal.
  arXiv  Detail & Related papers  (2024-02-06T20:38:46Z)
• DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions [27.489622263456983]
  We introduce DeceptPrompt, an algorithm that can generate adversarial natural language instructions that drive the Code LLMs to generate functionality correct code with vulnerabilities. When applying the optimized prefix/suffix, the attack success rate (ASR) will improve by average 50% compared with no prefix/suffix applying.
  arXiv  Detail & Related papers  (2023-12-07T22:19:06Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)
• DeepInception: Hypnotize Large Language Model to Be Jailbreaker [70.34096187718941]
  Large language models (LLMs) have succeeded significantly in various applications but remain susceptible to adversarial jailbreaks. We present a method to take advantage of the LLMs' personification capabilities to construct $textita virtual, nested scene. Empirically, the contents induced by our approach can achieve leading harmfulness rates with previous counterparts.
  arXiv  Detail & Related papers  (2023-11-06T15:29:30Z)
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks [99.23352758320945]
  We propose SmoothLLM, the first algorithm designed to mitigate jailbreaking attacks on large language models (LLMs) Based on our finding that adversarially-generated prompts are brittle to character-level changes, our defense first randomly perturbs multiple copies of a given input prompt, and then aggregates the corresponding predictions to detect adversarial inputs.
  arXiv  Detail & Related papers  (2023-10-05T17:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.