KryptoPilot: An Open-World Knowledge-Augmented LLM Agent for Automated Cryptographic Exploitation

• URL: http://arxiv.org/abs/2601.09129v1
• Date: Wed, 14 Jan 2026 04:02:40 GMT
• Title: KryptoPilot: An Open-World Knowledge-Augmented LLM Agent for Automated Cryptographic Exploitation
• Abstract summary: KryptoPilot is an open-world knowledge-augmented LLM agent for automated cryptographic exploitation.<n>We show that KryptoPilot achieves a complete solve rate on InterCode-CTF, solves between 56 and 60 percent of cryptographic challenges on the NYU-CTF benchmark, and successfully solves 26 out of 33 cryptographic challenges in live competitions.
• Score: 16.43451504898208
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Capture-the-Flag (CTF) competitions play a central role in modern cybersecurity as a platform for training practitioners and evaluating offensive and defensive techniques derived from real-world vulnerabilities. Despite recent advances in large language models (LLMs), existing LLM-based agents remain ineffective on high-difficulty cryptographic CTF challenges, which require precise cryptanalytic knowledge, stable long-horizon reasoning, and disciplined interaction with specialized toolchains. Through a systematic exploratory study, we show that insufficient knowledge granularity, rather than model reasoning capacity, is a primary factor limiting successful cryptographic exploitation: coarse or abstracted external knowledge often fails to support correct attack modeling and implementation. Motivated by this observation, we propose KryptoPilot, an open-world knowledge-augmented LLM agent for automated cryptographic exploitation. KryptoPilot integrates dynamic open-world knowledge acquisition via a Deep Research pipeline, a persistent workspace for structured knowledge reuse, and a governance subsystem that stabilizes reasoning through behavioral constraints and cost-aware model routing. This design enables precise knowledge alignment while maintaining efficient reasoning across heterogeneous subtasks. We evaluate KryptoPilot on two established CTF benchmarks and in six real-world CTF competitions. KryptoPilot achieves a complete solve rate on InterCode-CTF, solves between 56 and 60 percent of cryptographic challenges on the NYU-CTF benchmark, and successfully solves 26 out of 33 cryptographic challenges in live competitions, including multiple earliest-solved and uniquely-solved instances. These results demonstrate the necessity of open-world, fine-grained knowledge augmentation and governed reasoning for scaling LLM-based agents to real-world cryptographic exploitation.

Related papers

• CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild [49.44069582581181]
  CyberFactory transforms public vulnerability artifacts, including CVEs from the wild, into executable and verifiable task instances.<n>It uses a reusable vulnerability-analysis skill to guide the teacher through source inspection, problem solving with domain prior, and evidence-based validation.<n>On CyberGym, modelname reaches 52.4% Pass@1 under a one-hour budget, improving over its Qwen3.5 base model by +22.8 points.
  arXiv  Detail & Related papers  (2026-08-24T12:30:10Z)
• CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering [12.401873262343862]
  We study the cryptographic binary reverse engineering capabilities of large language models (LLMs)<n>Our benchmark comprises 432 challenges built from 48 standard cryptographic algorithms, 3 insecure crypto key usage scenarios, and 3 difficulty levels.<n>We also establish a strong human expert baseline of 92.19 points, showing that humans maintain an advantage in cryptographic RE tasks.
  arXiv  Detail & Related papers  (2026-04-04T14:51:09Z)
• Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs [65.6660735371212]
  We present textbftextscJustAsk, a framework that autonomously discovers effective extraction strategies through interaction alone.<n>It formulates extraction as an online exploration problem, using Upper Confidence Bound--based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration.<n>Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
  arXiv  Detail & Related papers  (2026-01-29T03:53:25Z)
• Quantigence: A Multi-Agent AI Framework for Quantum Security Research [0.0]
  Cryptographically Relevant Quantum Computers (CRQCs) pose a structural threat to the global digital economy.<n>We present Quantigence, a theory-driven multi-agent AI framework for structured quantum-security analysis.
  arXiv  Detail & Related papers  (2025-12-15T05:27:10Z)
• CryptoBench: A Dynamic Benchmark for Expert-Level Evaluation of LLM Agents in Cryptocurrency [60.83660377169452]
  This paper introduces CryptoBench, the first expert-curated, dynamic benchmark designed to rigorously evaluate the real-world capabilities of Large Language Model (LLM) agents.<n>Unlike general-purpose agent benchmarks for search and prediction, professional crypto analysis presents specific challenges.
  arXiv  Detail & Related papers  (2025-11-29T09:52:34Z)
• Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs [72.08224879435762]
  textttLearn-to-Ask is a simulator-free framework for learning and deploying proactive dialogue agents.<n>Our approach culminates in the successful deployment of LLMs into a live, large-scale online AI service.
  arXiv  Detail & Related papers  (2025-10-29T12:08:07Z)
• CTIArena: Benchmarking LLM Knowledge and Reasoning Across Heterogeneous Cyber Threat Intelligence [48.63397742510097]
  Cyber threat intelligence (CTI) is central to modern cybersecurity, providing critical insights for detecting and mitigating evolving threats.<n>With the natural language understanding and reasoning capabilities of large language models (LLMs), there is increasing interest in applying them to CTI.<n>We present CTIArena, the first benchmark for evaluating LLM performance on heterogeneous, multi-source CTI.
  arXiv  Detail & Related papers  (2025-10-13T22:10:17Z)
• WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning [73.91893534088798]
  WebSailor is a complete post-training methodology designed to instill this crucial capability.<n>Our approach involves generating novel, high-uncertainty tasks through structured sampling and information obfuscation.<n>WebSailor significantly outperforms all open-source agents in complex information-seeking tasks.
  arXiv  Detail & Related papers  (2025-09-16T17:57:03Z)
• AICrypto: A Comprehensive Benchmark for Evaluating Cryptography Capabilities of Large Language Models [31.974963309762913]
  AICrypto is the first comprehensive benchmark designed to evaluate the cryptography capabilities of large language models.<n>The benchmark comprises 135 multiple-choice questions, 150 capture-the-flag challenges, and 18 proof problems.<n>Our evaluation of 17 leading LLMs reveals that state-of-the-art models match or even surpass human experts in memorizing cryptographic concepts.
  arXiv  Detail & Related papers  (2025-07-13T11:11:01Z)
• WebSailor: Navigating Super-human Reasoning for Web Agent [72.5231321118689]
  WebSailor is a complete post-training methodology designed to instill this crucial capability.<n>Our approach involves generating novel, high-uncertainty tasks through structured sampling and information obfuscation.<n>WebSailor significantly outperforms all opensource agents in complex information-seeking tasks.
  arXiv  Detail & Related papers  (2025-07-03T12:59:07Z)
• CRAKEN: Cybersecurity LLM Agent with Knowledge-Based Execution [22.86304661035188]
  Large Language Model (LLM) agents can automate cybersecurity tasks and can adapt to the evolving cybersecurity landscape without re-engineering.<n>But they have two key limitations: accessing latest cybersecurity expertise beyond training data, and integrating new knowledge into complex task planning.<n>We present CRAKEN, a knowledge-based LLM agent framework that improves cybersecurity capability through three core mechanisms.
  arXiv  Detail & Related papers  (2025-05-21T11:01:11Z)
• Towards Explainable Vulnerability Detection with Large Language Models [14.243344783348398]
  Software vulnerabilities pose significant risks to the security and integrity of software systems.<n>The advent of large language models (LLMs) has introduced transformative potential due to their advanced generative capabilities.<n>In this paper, we propose LLMVulExp, an automated framework designed to specialize LLMs for the dual tasks of vulnerability detection and explanation.
  arXiv  Detail & Related papers  (2024-06-14T04:01:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.