An Ethically Grounded LLM-Based Approach to Insider Threat Synthesis and Detection

• URL: http://arxiv.org/abs/2509.06920v2
• Date: Wed, 01 Oct 2025 15:55:29 GMT
• Title: An Ethically Grounded LLM-Based Approach to Insider Threat Synthesis and Detection
• Abstract summary: This study introduces a novel, ethically grounded approach that uses the large language model (LLM) Claude Sonnet 3.7 to dynamically synthesize syslog messages.<n>The messages reflect real-world data distributions by being highly imbalanced (1% insider threats)<n>Sonnet 3.7 consistently outperformed GPT-4o across nearly all metrics, particularly in reducing false alarms and improving detection accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Insider threats are a growing organizational problem due to the complexity of identifying their technical and behavioral elements. A large research body is dedicated to the study of insider threats from technological, psychological, and educational perspectives. However, research in this domain has been generally dependent on datasets that are static and limited access which restricts the development of adaptive detection models. This study introduces a novel, ethically grounded approach that uses the large language model (LLM) Claude Sonnet 3.7 to dynamically synthesize syslog messages, some of which contain indicators of insider threat scenarios. The messages reflect real-world data distributions by being highly imbalanced (1% insider threats). The syslogs were analyzed for insider threats by both Sonnet 3.7 and GPT-4o, with their performance evaluated through statistical metrics including accuracy, precision, recall, F1, specificity, FAR, MCC, and ROC AUC. Sonnet 3.7 consistently outperformed GPT-4o across nearly all metrics, particularly in reducing false alarms and improving detection accuracy. The results show strong promise for the use of LLMs in synthetic dataset generation and insider threat detection.

Related papers

• SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring [45.617708056629766]
  We introduce SciHazard, a real-world-grounded benchmark for scientific risks and a dataset evaluation framework for measuring harmfulness.<n> SciHazard contains 2400 hazardous questions and 600 oversafety questions across 12 disciplines, with both queries grounded in regulated entities and documented failure scenarios.<n>TextscDeHarm-Score improves agreement with expert annotations by 90.17% over the strongest baseline.
  arXiv  Detail & Related papers  (2026-07-21T03:25:01Z)
• DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysis [13.9689104714669]
  DSIPA is a training-free framework that detects large language models (LLMs) content by quantifying sentiment distributional stability under controlled stylistic variation.<n>It operates in a zero-shot, black-box manner, leveraging two unsupervised metrics, sentiment distribution consistency and sentiment distribution preservation.<n>Our framework exhibits superior generalizability across domains and strong resilience to adversarial conditions.
  arXiv  Detail & Related papers  (2026-04-29T06:22:08Z)
• T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search [49.99459363244884]
  We propose a trajectory-aware evolutionary search method, T-MAP, which leverages execution trajectories to guide the discovery of adversarial prompts.<n>Our approach enables the automatic generation of attacks that not only bypass safety guardrails but also reliably realize harmful objectives through actual tool interactions.
  arXiv  Detail & Related papers  (2026-03-21T12:33:34Z)
• The Verification Crisis: Expert Perceptions of GenAI Disinformation and the Case for Reproducible Provenance [47.03825808787752]
  This article presents findings from the first wave of a longitudinal expert perception survey (N=21) involving AI researchers, policymakers, and disinformation specialists.<n>It examines the perceived severity of multimodal threats -- text, image, audio, and video -- and evaluates current mitigation strategies.<n>Results indicate that while deepfake video presents immediate "shock" value, large-scale text generation poses a systemic risk of "epistemic fragmentation"
  arXiv  Detail & Related papers  (2026-02-02T13:45:12Z)
• Semantic-Aware Advanced Persistent Threat Detection Using Autoencoders on LLM-Encoded System Logs [0.7611870296994722]
  Advanced Persistent Threats (APTs) are among the most challenging cyberattacks to detect.<n>Traditional statistical methods and shallow machine learning techniques often fail to detect them.<n>This paper proposes a novel anomaly detection approach that leverages semantic embeddings.
  arXiv  Detail & Related papers  (2026-01-30T12:38:12Z)
• Evaluating Large Language Models for Time Series Anomaly Detection in Aerospace Software [46.75681367373185]
  Time series anomaly detection (TSAD) is essential for ensuring the safety and reliability of aerospace software systems.<n>Large language models (LLMs) provide a promising training-free alternative to unsupervised approaches.<n>We introduce ATSADBench, the first benchmark for aerospace TSAD.
  arXiv  Detail & Related papers  (2026-01-18T15:07:16Z)
• DarkPatterns-LLM: A Multi-Layer Benchmark for Detecting Manipulative and Harmful AI Behavior [0.0]
  We introduce textbfDarkPatterns-LLM, a comprehensive benchmark dataset and diagnostic framework for fine-grained assessment of manipulative content.<n>Our framework implements a four-layer analytical pipeline comprising Multi-Granular Detection (MGD), Multi-Scale Intent Analysis (MSIAN), Threat Harmonization Protocol (THP) and Deep Contextual Risk Alignment (DCRA)<n>The dataset contains 401 meticulously curated examples with instruction-response pairs and expert annotations.
  arXiv  Detail & Related papers  (2025-12-27T05:05:46Z)
• From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs [65.04549036809557]
  We introduce a benchmark built from pedestrian-perspective videos captured with synchronized stereo cameras, LiDAR, and IMU/GPS sensors.<n>This dataset provides metrically precise 3D information, enabling the automatic generation of spatial reasoning questions.<n> Evaluations reveal that the performance gains observed in structured indoor benchmarks vanish in open-world settings.
  arXiv  Detail & Related papers  (2025-12-22T18:58:12Z)
• AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research [81.04845910798387]
  Generating natural language explanations for threat detections remains an open problem in cybersecurity research.<n>We present AutoMalDesc, an automated static analysis summarization framework that operates independently at scale.<n>We publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) datasets, along with our methodology and evaluation framework.
  arXiv  Detail & Related papers  (2025-11-17T13:05:25Z)
• Subliminal Corruption: Mechanisms, Thresholds, and Interpretability [0.6875312133832079]
  Subliminal corruption is transmitted through semantically neutral data, bypassing standard safety checks.<n>We present a systematic study of the scaling laws, thresholds, and mechanisms of subliminal corruption using a teacher-student setup with GPT-2.<n>Results demonstrate a critical vulnerability in AI systems that rely on synthetic data and highlight the need for new safety protocols.
  arXiv  Detail & Related papers  (2025-10-22T00:57:51Z)
• An Automated Attack Investigation Approach Leveraging Threat-Knowledge-Augmented Large Language Models [17.220143037047627]
  Advanced Persistent Threats (APTs) compromise high-value systems to steal data or disrupt operations.<n>Existing methods suffer from poor platform generality, limited generalization to evolving tactics, and an inability to produce analyst-ready reports.<n>We present an LLM-empowered attack investigation framework augmented with a dynamically adaptable Kill-Chain-aligned threat knowledge base.
  arXiv  Detail & Related papers  (2025-09-01T08:57:01Z)
• Chimera: Harnessing Multi-Agent LLMs for Automatic Insider Threat Simulation [17.496651394447596]
  We propose Chimera, the first large language model (LLM)-based multi-agent framework that automatically simulates both benign and malicious insider activities.<n>Chimera models each employee with agents that have role-specific behavior and integrates modules for group meetings, pairwise interactions, and autonomous scheduling.<n>It incorporates 15 types of insider attacks (e.g., IP theft, system sabotage) and has been deployed to simulate activities in three sensitive domains.<n>We assess ChimeraLog via human studies and quantitative analysis, confirming its diversity, realism, and presence of explainable threat patterns.
  arXiv  Detail & Related papers  (2025-08-11T08:24:48Z)
• Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection [44.05134959039957]
  We investigate how sociolinguistic attributes-gender, CEFR proficiency, academic field, and language environment-impact state-of-the-art AI text detectors.<n>Our results reveal significant biases: CEFR proficiency and language environment consistently affected detector accuracy, while gender and academic field showed detector-dependent effects.<n>These findings highlight the crucial need for socially aware AI text detection to avoid unfairly penalizing specific demographic groups.
  arXiv  Detail & Related papers  (2025-02-18T07:49:31Z)
• The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility? [54.18519360412294]
  Large Language Models (LLMs) must balance between rejecting harmful requests for safety and accommodating legitimate ones for utility.<n>This paper presents a Direct Preference Optimization (DPO) based alignment framework that achieves better overall performance.<n>We analyze experimental results obtained from testing DeepSeek-R1 on our benchmark and reveal the critical ethical concerns raised by this highly acclaimed model.
  arXiv  Detail & Related papers  (2025-01-20T06:35:01Z)
• Evaluating ML Robustness in GNSS Interference Classification, Characterization & Localization [42.14439854721613]
  Jamming devices disrupt signals from the global navigation satellite system (GNSS)<n>This paper introduces an extensive dataset comprising snapshots obtained from a low-frequency antenna.<n>Our objective is to assess the resilience of machine learning (ML) models against environmental changes.
  arXiv  Detail & Related papers  (2024-09-23T15:20:33Z)
• Enhancing Robustness of LLM-Synthetic Text Detectors for Academic Writing: A Comprehensive Analysis [35.351782110161025]
  Large language models (LLMs) offer numerous advantages in terms of revolutionizing work and study methods. They have also garnered significant attention due to their potential negative consequences. One example is generating academic reports or papers with little to no human contribution.
  arXiv  Detail & Related papers  (2024-01-16T01:58:36Z)
• Efficacy of Utilizing Large Language Models to Detect Public Threat Posted Online [0.0]
  This paper examines the efficacy of utilizing large language models (LLMs) to detect public threats posted online. Data collection tools were developed to amass post titles from a popular Korean online community. GPT-4 performed best overall with 97.9% non-threat and 100% threat accuracy.
  arXiv  Detail & Related papers  (2023-12-29T16:42:02Z)
• Large Language Models can be Guided to Evade AI-Generated Text Detection [40.7707919628752]
  Large language models (LLMs) have shown remarkable performance in various tasks and have been extensively utilized by the public. We equip LLMs with prompts, rather than relying on an external paraphraser, to evaluate the vulnerability of these detectors. We propose a novel Substitution-based In-Context example optimization method (SICO) to automatically construct prompts for evading the detectors.
  arXiv  Detail & Related papers  (2023-05-18T10:03:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.