SecureBERT 2.0: Advanced Language Model for Cybersecurity Intelligence

• URL: http://arxiv.org/abs/2510.00240v2
• Date: Fri, 10 Oct 2025 19:38:53 GMT
• Title: SecureBERT 2.0: Advanced Language Model for Cybersecurity Intelligence
• Authors: Ehsan Aghaei, Sarthak Jain, Prashanth Arun, Arjun Sambamoorthy,
• Abstract summary: We present SecureBERT 2.0, an enhanced encoder-only language model purpose-built for cybersecurity applications.<n>It introduces improved long-context modeling and hierarchical encoding, enabling effective processing of extended and heterogeneous documents.<n>SecureBERT 2.0 achieves state-of-the-art performance on multiple cybersecurity benchmarks.
• Score: 5.844061361572245
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective analysis of cybersecurity and threat intelligence data demands language models that can interpret specialized terminology, complex document structures, and the interdependence of natural language and source code. Encoder-only transformer architectures provide efficient and robust representations that support critical tasks such as semantic search, technical entity extraction, and semantic analysis, which are key to automated threat detection, incident triage, and vulnerability assessment. However, general-purpose language models often lack the domain-specific adaptation required for high precision. We present SecureBERT 2.0, an enhanced encoder-only language model purpose-built for cybersecurity applications. Leveraging the ModernBERT architecture, SecureBERT 2.0 introduces improved long-context modeling and hierarchical encoding, enabling effective processing of extended and heterogeneous documents, including threat reports and source code artifacts. Pretrained on a domain-specific corpus more than thirteen times larger than its predecessor, comprising over 13 billion text tokens and 53 million code tokens from diverse real-world sources, SecureBERT 2.0 achieves state-of-the-art performance on multiple cybersecurity benchmarks. Experimental results demonstrate substantial improvements in semantic search for threat intelligence, semantic analysis, cybersecurity-specific named entity recognition, and automated vulnerability detection in code within the cybersecurity domain.

Related papers

• CIBER: A Comprehensive Benchmark for Security Evaluation of Code Interpreter Agents [27.35968236632966]
  LLM-based code interpreter agents are increasingly deployed in critical situations.<n>Existing benchmarks fail to capture the security risks arising from dynamic code execution, tool interactions, and multi-turn context.<n>We introduce CIBER, an automated benchmark that combines dynamic attack generation, isolated secure sandboxing, and state-aware evaluation.
  arXiv  Detail & Related papers  (2026-02-23T06:41:41Z)
• Cyber Threat Detection and Vulnerability Assessment System using Generative AI and Large Language Model [0.0]
  Cyber-attacks include various threats such as ransomware, malware, phishing, and Denial of Service (DoS)-related attacks.<n>Traditional models such as Gene Artificial Intelligence (AI) and Security Bidirectional Representations from Transformers (BERT) were implemented to detect cyber threats.<n>The existing Security BERT model has a limited contextual understanding of text data, which has less impact on detecting cyber-attacks.<n>The proposed RoBERTa model achieved better results than the existing BERT model in terms of accuracy (0.99), recall (0.91), and precision (0.89) respectively.
  arXiv  Detail & Related papers  (2026-01-08T19:19:54Z)
• The Application of Transformer-Based Models for Predicting Consequences of Cyber Attacks [0.4604003661048266]
  Threat Modeling can provide critical support to cybersecurity professionals, enabling them to take timely action and allocate resources that could be used elsewhere.<n>Recently, there has been a pressing need for automated methods to assess attack descriptions and forecast the future consequences of cyberattacks.<n>This study examines how Natural Language Processing (NLP) and deep learning can be applied to analyze the potential impact of cyberattacks.
  arXiv  Detail & Related papers  (2025-08-18T15:46:36Z)
• Decompiling Smart Contracts with a Large Language Model [51.49197239479266]
  Despite Etherscan's 78,047,845 smart contracts deployed on (as of May 26, 2025), a mere 767,520 ( 1%) are open source.<n>This opacity necessitates the automated semantic analysis of on-chain smart contract bytecode.<n>We introduce a pioneering decompilation pipeline that transforms bytecode into human-readable and semantically faithful Solidity code.
  arXiv  Detail & Related papers  (2025-06-24T13:42:59Z)
• CTINexus: Automatic Cyber Threat Intelligence Knowledge Graph Construction Using Large Language Models [49.657358248788945]
  Textual descriptions in cyber threat intelligence (CTI) reports are rich sources of knowledge about cyber threats.<n>Current CTI knowledge extraction methods lack flexibility and generalizability.<n>We propose CTINexus, a novel framework for data-efficient CTI knowledge extraction and high-quality cybersecurity knowledge graph (CSKG) construction.
  arXiv  Detail & Related papers  (2024-10-28T14:18:32Z)
• ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models [65.79770974145983]
  ASSERT, Automated Safety Scenario Red Teaming, consists of three methods -- semantically aligned augmentation, target bootstrapping, and adversarial knowledge injection. We partition our prompts into four safety domains for a fine-grained analysis of how the domain affects model performance. We find statistically significant performance differences of up to 11% in absolute classification accuracy among semantically related scenarios and error rates of up to 19% absolute error in zero-shot adversarial settings.
  arXiv  Detail & Related papers  (2023-10-14T17:10:28Z)
• Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices [3.340416780217405]
  This paper presents SecurityBERT, a novel architecture that leverages the Bidirectional Representations from Transformers (BERT) model for cyber threat detection in IoT networks. Our research demonstrates that SecurityBERT outperforms traditional Machine Learning (ML) and Deep Learning (DL) methods, such as Convolutional Neural Networks (CNNIoTs) or Recurrent Neural Networks (IoTRNNs) in cyber threat detection. SecurityBERT achieved an impressive 98.2% overall accuracy in identifying fourteen distinct attack types, surpassing previous records set by hybrid solutions.
  arXiv  Detail & Related papers  (2023-06-25T15:04:21Z)
• Multi-features based Semantic Augmentation Networks for Named Entity Recognition in Threat Intelligence [7.321994923276344]
  We propose a semantic augmentation method which incorporates different linguistic features to enrich the representation of input tokens. In particular, we encode and aggregate the constituent feature, morphological feature and part of speech feature for each input token to improve the robustness of the method. We have conducted experiments on the cybersecurity datasets DNRTI and MalwareTextDB, and the results demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-07-01T06:55:12Z)
• A Deep Learning Approach for Ontology Enrichment from Unstructured Text [2.932750332087746]
  Existing information vulnerabilities on attacks, controls, and advisories available on the web provide an opportunity to represent and perform security analytics. Ontology enrichment algorithms based on natural language processing and ML models have issues with contextual extraction of concepts in words, phrases, and sentences. Bidirectional LSTMs trained on a large DB dataset and Wikipedia corpus of 2.8 GB along with Universal Sentence is deployed to enrich ISO-based information security.
  arXiv  Detail & Related papers  (2021-12-16T01:32:21Z)
• OntoEnricher: A Deep Learning Approach for Ontology Enrichment from Unstructured Text [2.707154152696381]
  Existing information on vulnerabilities, controls, and advisories available on the web provides an opportunity to represent knowledge and perform analytics to mitigate some of the concerns. This necessitates dynamic and automated enrichment of information security. Existing ontology enrichment algorithms based on natural processing and ML models have issues with the contextual extraction of concepts in words, phrases and sentences.
  arXiv  Detail & Related papers  (2021-02-08T09:43:05Z)
• A System for Efficiently Hunting for Cyber Threats in Computer Systems Using Threat Intelligence [78.23170229258162]
  We build ThreatRaptor, a system that facilitates cyber threat hunting in computer systems using OSCTI. ThreatRaptor provides (1) an unsupervised, light-weight, and accurate NLP pipeline that extracts structured threat behaviors from unstructured OSCTI text, (2) a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities, and (3) a query synthesis mechanism that automatically synthesizes a TBQL query from the extracted threat behaviors.
  arXiv  Detail & Related papers  (2021-01-17T19:44:09Z)
• Enabling Efficient Cyber Threat Hunting With Cyber Threat Intelligence [94.94833077653998]
  ThreatRaptor is a system that facilitates threat hunting in computer systems using open-source Cyber Threat Intelligence (OSCTI) It extracts structured threat behaviors from unstructured OSCTI text and uses a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities. Evaluations on a broad set of attack cases demonstrate the accuracy and efficiency of ThreatRaptor in practical threat hunting.
  arXiv  Detail & Related papers  (2020-10-26T14:54:01Z)
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective [84.78604733927887]
  Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks. Recent studies show that such BERT-based models are vulnerable facing the threats of textual adversarial attacks. We propose InfoBERT, a novel learning framework for robust fine-tuning of pre-trained language models.
  arXiv  Detail & Related papers  (2020-10-05T20:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.