CyBOKClaw: Human-in-the-Loop CyBOK Mapping for Cybersecurity Curriculum

• URL: http://arxiv.org/abs/2605.24663v1
• Date: Sat, 23 May 2026 17:04:31 GMT
• Title: CyBOKClaw: Human-in-the-Loop CyBOK Mapping for Cybersecurity Curriculum
• Abstract summary: CyBOKClaw is a retrieval framework for mapping cybersecurity keywords to the Cyber Security Body of Knowledge (CyBOK)<n>It combines query normalization, curated term expansion, concept-level boosts, topic-description enrichment, and domain-sensitive ranking rules.<n>We evaluate the framework using both structural retrieval metrics and an expert-guided top-5 usefulness metric, ECA-5.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents CyBOKClaw, an interpretable human-in-the-loop retrieval framework for mapping cybersecurity keywords or phrases (KWoPs) to the Cyber Security Body of Knowledge (CyBOK). Rather than treating the task as strict exact classification, the framework is designed as a top-k candidate generator for expert review. It combines query normalization, curated term expansion, concept-level boosts, topic-description enrichment, and domain-sensitive ranking rules. Because educational KWoPs are often broad, ambiguous, and only approximately aligned with CyBOK terminology, strict exact matching provides only a partial account of practical utility. We therefore evaluate the framework using both structural retrieval metrics and an expert-guided top-5 usefulness metric, ECA-5 (Exact or Closest Acceptable Match at top-5), which records whether the returned candidates contain at least one mapping that an expert would judge exact or accept as the nearest practical CyBOK placement. On the development dataset, CyBOKClaw achieves 64.73% EXA-5 (Exact Match at top-5), 84.18% structural semantic alignment, and 91.88% ECA-5; on the validation dataset, it achieves 81.19% EXA-5, 93.32% structural semantic alignment, and 98.00% ECA-5. These results show that expert-guided top-k usefulness provides a more faithful account of practical CyBOK mapping utility than exact structural matching alone, and that CyBOKClaw is effective as a CyBOK-specific expert-support retrieval system.

Related papers

• TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation [74.69068352546073]
  TRACE-Bench is an evaluation tool for multi-reference images spanning diverse artistic styles and real-world subjects.<n>It is built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects.<n>Its formula structure drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis.
  arXiv  Detail & Related papers  (2026-08-17T16:15:50Z)
• ECG-InterpBench: Benchmarking the Interpretability of ECG Foundation Models with Matched-Scale Sparse Autoencoders [0.8162386935076237]
  ECG-InterpBench is a benchmark designed to evaluate the interpretability of ECG foundation-model representations.<n>We evaluate six frozen ECG foundation models across five standardized depths, five matched dictionary widths, and three random seeds.<n>The benchmark reveals that ECG foundation models exhibit distinct interpretability profiles.
  arXiv  Detail & Related papers  (2026-07-29T19:20:50Z)
• Context-Aware Distillation and Ablation for Text2DSL [78.29352305480285]
  We replace prompt-only synthetic generation with context-aware distillation.<n>A teacher large language model operates under an explicitly defined structured context.<n>This scales the verified PolkitBench corpus from 4,204 to 10,073 pairs.
  arXiv  Detail & Related papers  (2026-06-21T16:27:24Z)
• Novelty-Aware Agentic Retrieval: Comparing Research Contributions Through Structured Multi-Step Reasoning [0.0]
  Novelty-Aware Research Agent is a prototype agentic retrieval system.<n>It layers structured multi-step reasoning on a RAG pipeline through six typed-contract components.<n>It produces structured comparison artifacts: per-paper contribution records, paper-level overlaps, and a problem x method gap matrix.
  arXiv  Detail & Related papers  (2026-06-20T17:04:02Z)
• SkillDAG: Self-Evolving Typed Skill Graphs for LLM Skill Selection at Scale [54.70985426016736]
  We present SkillDAG, which models inter-skill relationships as a typed directed graph.<n>Each search returns vector matches, typed-edge neighbors, and conflict signals, and a propose-then-commit protocol lets the agent register execution-backed edges.<n>On ALFWorld and SkillsBench with MiniMax-M2.7, SkillDAG reaches 67.1% success and 27.3% reward.
  arXiv  Detail & Related papers  (2026-06-02T02:45:21Z)
• Encyclo-K: Evaluating LLMs with Dynamically Composed Knowledge Statements [78.87065404966002]
  Existing benchmarks predominantly curate questions at the question level.<n>We propose Encyclo-K, a statement-based benchmark that rethinks benchmark construction from the ground up.
  arXiv  Detail & Related papers  (2025-12-31T13:55:54Z)
• BRIDGE: Building Representations In Domain Guided Program Verification [67.36686119518441]
  BRIDGE decomposes verification into three interconnected domains: Code, Specifications, and Proofs.<n>We show that this approach substantially improves both accuracy and efficiency beyond standard error feedback methods.
  arXiv  Detail & Related papers  (2025-11-26T06:39:19Z)
• Beyond Long Context: When Semantics Matter More than Tokens [0.0]
  Electronic Health Records store clinical documentation as base64 attachments encoded in FHIR DocumentReference resources.<n>The Clinical Entity Augmented Retrieval (CLEAR) method, introduced by Lopez et al. 2025, uses entity aware retrieval.<n>CLEAR achieved a 58.3 percent win rate, an average semantic similarity of 0.878, and used 78 percent fewer tokens than wide context processing.
  arXiv  Detail & Related papers  (2025-10-29T16:41:44Z)
• Hierarchical Conformal Classification [5.964388602612373]
  Conformal prediction (CP) is a powerful framework for quantifying uncertainty in machine learning models.<n>Standard CP treats classes as flat and unstructured, ignoring relationships such as semantic or hierarchical structure among class labels.<n>This paper presents hierarchical HCCal classification (HCC), an extension of CP that incorporates class hierarchies into both the structure and semantics of prediction sets.
  arXiv  Detail & Related papers  (2025-08-18T18:05:55Z)
• Advancing Retrieval-Augmented Generation for Structured Enterprise and Internal Data [0.0]
  Large Language Models (LLMs) have strong generative capabilities.<n>They are limited by static pretraining, short context windows, and challenges in processing heterogeneous data formats.<n>Conventional Retrieval-Augmented Generation (RAG) frameworks address some of these gaps but often struggle with structured and semi-structured data.<n>This work proposes an advanced RAG framework that combines hybrid retrieval strategies using dense embeddings (all-mpnet-base-v2) and BM25, enhanced by metadata-aware filtering with SpaCy NER and cross-encoder reranking.
  arXiv  Detail & Related papers  (2025-07-16T17:13:06Z)
• CLIP-EBC: CLIP Can Count Accurately through Enhanced Blockwise Classification [29.570935239024138]
  We propose CLIP-EBC, the first fully CLIP-based model for accurate crowd density estimation.<n>Our EBC framework can improve existing classification-based methods by up to 44.5% on the UCF-QNRF dataset.<n>CLIP-EBC achieves state-of-the-art performance on the NWPU-Crowd test set, with an MAE of 58.2 and an RMSE of 268.5, representing improvements of 8.6% and 13.3% over the previous best method, STEERER.
  arXiv  Detail & Related papers  (2024-03-14T11:08:33Z)
• Investigating the Limitation of CLIP Models: The Worst-Performing Categories [53.360239882501325]
  Contrastive Language-Image Pre-training (CLIP) provides a foundation model by integrating natural language into visual concepts. It is usually expected that satisfactory overall accuracy can be achieved across numerous domains through well-designed textual prompts. However, we found that their performance in the worst categories is significantly inferior to the overall performance.
  arXiv  Detail & Related papers  (2023-10-05T05:37:33Z)
• Optimizing Partial Area Under the Top-k Curve: Theory and Practice [151.5072746015253]
  We develop a novel metric named partial Area Under the top-k Curve (AUTKC) AUTKC has a better discrimination ability, and its Bayes optimal score function could give a correct top-K ranking with respect to the conditional probability. We present an empirical surrogate risk minimization framework to optimize the proposed metric.
  arXiv  Detail & Related papers  (2022-09-03T11:09:13Z)
• Inductive Learning on Commonsense Knowledge Graph Completion [89.72388313527296]
  Commonsense knowledge graph (CKG) is a special type of knowledge graph (CKG) where entities are composed of free-form text. We propose to study the inductive learning setting for CKG completion where unseen entities may present at test time. InductivE significantly outperforms state-of-the-art baselines in both standard and inductive settings on ATOMIC and ConceptNet benchmarks.
  arXiv  Detail & Related papers  (2020-09-19T16:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.