Improving LLM-Based SSH Honeypots Through Prompting and Fine-Tuning

• URL: http://arxiv.org/abs/2608.18686v1
• Date: Wed, 19 Aug 2026 08:35:04 GMT
• Title: Improving LLM-Based SSH Honeypots Through Prompting and Fine-Tuning
• Abstract summary: This paper studies how to improve and evaluate the shell emulation accuracy of local LLM-based SSH honeypots.<n>We find that prompt design has a large effect and that fine-tuning depends on dataset coverage.
• Score: 0.1749935196721634
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: LLM-based SSH honeypots often use closed cloud LLMs because they give strong shell realism, but cloud models create deployment problems. These include no stable versioning, provider-side changes, attacker-driven cost, and model decommissioning. Local open-weight models avoid these problems, but they usually perform worse and make mistakes that reveal the honeypot. These mistakes include malformed outputs, command echoing, inconsistent filesystem state, and AI-style artifacts. This paper studies how to improve and evaluate the shell emulation accuracy of local LLM-based SSH honeypots using prompt design and supervised fine-tuning. We fine-tune and evaluate eight models in total: the original fine-tuned GPT-3.5 model used in shelLM and seven open-weight local models, each compared to its base model. We also test how prompt structure transfers across model families. Using 34 automated unit tests that measure shell emulation accuracy in single-session and fresh-session settings, we find that prompt design has a large effect and that fine-tuning depends on dataset coverage. Fine-tuning on the original 112-conversation dataset does not improve aggregate pass rate, while an expanded dataset built from honeypot logs produces clearly stronger local models. Taken together, the results suggest that prompting and fine-tuning can each improve local LLM honeypots on their own, but their effects do not combine straightforwardly, since strong rule-based prompting and supervised adaptation can also conflict by addressing overlapping shell-behavior constraints.

Related papers

• MERGETUNE: Continued fine-tuning of vision-language models [77.8627788911249]
  Fine-tuning vision-language models (VLMs) often leads to catastrophic forgetting of pretrained knowledge.<n>We introduce a novel paradigm, continued fine-tuning (CFT), which seeks to recover pretrained knowledge after a zero-shot model has already been adapted.
  arXiv  Detail & Related papers  (2026-01-15T15:15:53Z)
• PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise [60.63315470285562]
  MiniTruePrefixes is a novel specialized model that better detects factual inconsistencies over text prefixes.<n>We show that integrating MiniTruePrefixes into a controlled decoding framework substantially improves factual consistency in abstractive summarization.
  arXiv  Detail & Related papers  (2025-11-03T09:07:44Z)
• LLMHoney: A Real-Time SSH Honeypot with Large Language Model-Driven Dynamic Response Generation [0.0]
  This Report presents LLMHoney, an SSH honeypot that leverages Large Language Models (LLMs) to generate realistic, dynamic command outputs in real time.<n>LLMs integrate a dictionary-based virtual file system to handle common commands with low latency.<n>We report comprehensive metrics including accuracy (exact-match, Cosine Similarity, Jaro-Winkler Similarity, Levenshtein Similarity and BLEU score), response latency and memory overhead.
  arXiv  Detail & Related papers  (2025-09-01T13:28:00Z)
• Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications [5.713077600587505]
  We use large language models (LLMs) to synthesize a high-quality dataset of error correction pairs.<n>We then adapt the synthetic data distribution to match the mobile application domain by reweighting the samples.<n>We present best practices for mixing our synthetic data with other data sources to improve model performance on error correction in both offline evaluation and production live A/B testing.
  arXiv  Detail & Related papers  (2025-05-24T03:27:20Z)
• S$^2$R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning [51.84977135926156]
  We introduce S$2$R, an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference.<n>Our results demonstrate that Qwen2.5-math-7B achieves an accuracy improvement from 51.0% to 81.6%, outperforming models trained on an equivalent amount of long-CoT distilled data.
  arXiv  Detail & Related papers  (2025-02-18T13:40:22Z)
• Pruning Foundation Models for High Accuracy without Retraining [48.256389781305415]
  It is challenging to deploy foundation models or large language models (LLMs) due to their massive parameters and computations. Post-training pruning methods are proposed to prune LLMs in one-shot without retraining. Our experiments demonstrate the superior performance of the proposed methods in comparison to SOTA baselines.
  arXiv  Detail & Related papers  (2024-10-21T01:23:34Z)
• ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA [55.697627106315004]
  Large language models (LLMs) require model editing to efficiently update specific knowledge within them and avoid factual errors.<n>Previous approaches manage sequential edits by freezing original parameters and discretely allocating new parameters for each knowledge update.<n>We propose ELDER, a novel approach to create a continuous association between data and adapters.
  arXiv  Detail & Related papers  (2024-08-19T02:27:00Z)
• Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models [0.8399688944263842]
  Large Language Models (LLMs) have the capability to understand and generate human-like text from input queries. This study extends this concept to the integration of LLMs within Retrieval-Augmented Generation (RAG) pipelines. We evaluate the impact of fine-tuning on the LLMs' capacity for data extraction and contextual understanding.
  arXiv  Detail & Related papers  (2024-06-17T04:35:17Z)
• MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents [62.02920842630234]
  We show how to build small fact-checking models that have GPT-4-level performance but for 400x lower cost. We do this by constructing synthetic training data with GPT-4, which involves creating realistic yet challenging instances of factual errors. For evaluation, we unify datasets from recent work on fact-checking and grounding LLM generations into a new benchmark, LLM-AggreFact.
  arXiv  Detail & Related papers  (2024-04-16T17:59:10Z)
• Automated Data Curation for Robust Language Model Fine-Tuning [13.8454385440986]
  We introduce an automated data curation pipeline CLEAR for instruction tuning datasets. CLEAR estimates which training data is low-quality and either filters or corrects it. Experiments reveal that CLEAR consistently improves the performance of fine-tuned models across many datasets and models.
  arXiv  Detail & Related papers  (2024-03-19T14:44:45Z)
• CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model [22.870512676002463]
  This paper focuses on Offsite-Tuning (OFT), a representative technique that transfers transformer blocks between centralized LLMs and downstream emulators. Inspired by these observations, we propose CRaSh, involving Clustering, Removing, and Sharing, a training-free strategy to derive improved emulators from LLMs. Our findings demonstrate a linear connectivity among these optima falling over the same basin, thereby highlighting the effectiveness of CRaSh and OFT.
  arXiv  Detail & Related papers  (2023-10-24T03:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.