LLM4Fuzz: Guided Fuzzing of Smart Contracts with Large Language Models

• URL: http://arxiv.org/abs/2401.11108v1
• Date: Sat, 20 Jan 2024 04:07:53 GMT
• Title: LLM4Fuzz: Guided Fuzzing of Smart Contracts with Large Language Models
• Authors: Chaofan Shou, Jing Liu, Doudou Lu, Koushik Sen
• Abstract summary: This paper introduces LLM4Fuzz to optimize automated smart contract security analysis. It uses large language models (LLMs) to intelligently guide and prioritize fuzzing campaigns. Evaluations show substantial gains in efficiency, coverage, and vulnerability detection.
• Score: 7.833199151422389
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As blockchain platforms grow exponentially, millions of lines of smart contract code are being deployed to manage extensive digital assets. However, vulnerabilities in this mission-critical code have led to significant exploitations and asset losses. Thorough automated security analysis of smart contracts is thus imperative. This paper introduces LLM4Fuzz to optimize automated smart contract security analysis by leveraging large language models (LLMs) to intelligently guide and prioritize fuzzing campaigns. While traditional fuzzing suffers from low efficiency in exploring the vast state space, LLM4Fuzz employs LLMs to direct fuzzers towards high-value code regions and input sequences more likely to trigger vulnerabilities. Additionally, LLM4Fuzz can leverage LLMs to guide fuzzers based on user-defined invariants, reducing blind exploration overhead. Evaluations of LLM4Fuzz on real-world DeFi projects show substantial gains in efficiency, coverage, and vulnerability detection compared to baseline fuzzing. LLM4Fuzz also uncovered five critical vulnerabilities that can lead to a loss of more than $247k.

Related papers

• Understanding LLMs' Fluid Intelligence Deficiency: An Analysis of the ARC Task [71.61879949813998]
  In cognitive research, the latter ability is referred to as fluid intelligence, which is considered to be critical for assessing human intelligence. Recent research on fluid intelligence assessments has highlighted significant deficiencies in LLMs' abilities. Our study revealed three major limitations in existing LLMs: limited ability for skill composition, unfamiliarity with abstract input formats, and the intrinsic deficiency of left-to-right decoding.
  arXiv  Detail & Related papers  (2025-02-11T02:31:09Z)
• From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security [0.0]
  Large Language Models (LLMs) have emerged as powerful tools for automating various programming tasks. LLMs could introduce vulnerabilities unbeknown to the programmer. When analyzing code, they could miss clear vulnerabilities or signal nonexistent ones.
  arXiv  Detail & Related papers  (2024-12-19T16:20:22Z)
• What You See Is Not Always What You Get: An Empirical Study of Code Comprehension by Large Language Models [0.5735035463793009]
  We investigate the vulnerability of large language models (LLMs) to imperceptible attacks, where hidden character manipulation in source code misleads LLMs' behaviour while remaining undetectable to human reviewers. These attacks include coding reordering, invisible coding characters, code deletions, and code homoglyphs. Our findings confirm the susceptibility of LLMs to imperceptible coding character attacks, while different LLMs present different negative correlations between perturbation magnitude and performance.
  arXiv  Detail & Related papers  (2024-12-11T04:52:41Z)
• Smart-LLaMA: Two-Stage Post-Training of Large Language Models for Smart Contract Vulnerability Detection and Explanation [21.39496709865097]
  Existing smart contract vulnerability detection methods face three main issues. Insufficient quality of datasets, lacking detailed explanations and precise vulnerability locations. We propose Smart-LLaMA, an advanced detection method based on the LLaMA language model.
  arXiv  Detail & Related papers  (2024-11-09T15:49:42Z)
• Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities [63.603861880022954]
  We introduce ADV-LLM, an iterative self-tuning process that crafts adversarial LLMs with enhanced jailbreak ability. Our framework significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs. It exhibits strong attack transferability to closed-source models, achieving 99% ASR on GPT-3.5 and 49% ASR on GPT-4, despite being optimized solely on Llama3.
  arXiv  Detail & Related papers  (2024-10-24T06:36:12Z)
• GIVE: Structured Reasoning of Large Language Models with Knowledge Graph Inspired Veracity Extrapolation [108.2008975785364]
  Graph Inspired Veracity Extrapolation (GIVE) is a novel reasoning method that merges parametric and non-parametric memories to improve accurate reasoning with minimal external input. GIVE guides the LLM agent to select the most pertinent expert data (observe), engage in query-specific divergent thinking (reflect), and then synthesize this information to produce the final output (speak)
  arXiv  Detail & Related papers  (2024-10-11T03:05:06Z)
• ANVIL: Anomaly-based Vulnerability Identification without Labelled Training Data [5.938113434208745]
  Supervised learning-based software vulnerability detectors often fall short due to the inadequate availability of labelled training data. In this paper, we explore a different approach by reframing vulnerability detection as one of anomaly detection. Our approach achieves $1.62times$ to $2.18times$ better Top-5 accuracies and $1.02times$ to $1.29times$ times better ROC scores on line-level vulnerability detection tasks.
  arXiv  Detail & Related papers  (2024-08-28T03:28:17Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models [35.77228114378362]
  Large Language Models (LLMs) generate malicious outputs when inputs contain specific "triggers" set by attackers. Traditional defense strategies are impractical for API-accessible LLMs due to limited model access, high computational costs, and data requirements. We propose Chain-of-Scrutiny (CoS) which leverages LLMs' unique reasoning abilities to mitigate backdoor attacks.
  arXiv  Detail & Related papers  (2024-06-10T00:53:25Z)
• CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models [6.931433424951554]
  Large language models (LLMs) introduce new security risks, but there are few comprehensive evaluation suites to measure and reduce these risks. We present BenchmarkName, a novel benchmark to quantify LLM security risks and capabilities. We evaluate multiple state-of-the-art (SOTA) LLMs, including GPT-4, Mistral, Meta Llama 3 70B-Instruct, and Code Llama.
  arXiv  Detail & Related papers  (2024-04-19T20:11:12Z)
• An Insight into Security Code Review with LLMs: Capabilities, Obstacles and Influential Factors [9.309745288471374]
  Security code review is a time-consuming and labor-intensive process. Existing security analysis tools struggle with poor generalization, high false positive rates, and coarse detection granularity. Large Language Models (LLMs) have been considered promising candidates for addressing those challenges.
  arXiv  Detail & Related papers  (2024-01-29T17:13:44Z)
• How Far Have We Gone in Vulnerability Detection Using Large Language Models [15.09461331135668]
  We introduce a comprehensive vulnerability benchmark VulBench. This benchmark aggregates high-quality data from a wide range of CTF challenges and real-world applications. We find that several LLMs outperform traditional deep learning approaches in vulnerability detection.
  arXiv  Detail & Related papers  (2023-11-21T08:20:39Z)
• Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs [59.596335292426105]
  This paper collects the first open-source dataset to evaluate safeguards in large language models. We train several BERT-like classifiers to achieve results comparable with GPT-4 on automatic safety evaluation.
  arXiv  Detail & Related papers  (2023-08-25T14:02:12Z)
• On Evaluating Adversarial Robustness of Large Vision-Language Models [64.66104342002882]
  We evaluate the robustness of large vision-language models (VLMs) in the most realistic and high-risk setting. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP. Black-box queries on these VLMs can further improve the effectiveness of targeted evasion.
  arXiv  Detail & Related papers  (2023-05-26T13:49:44Z)
• Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization [66.08074487429477]
  We investigate the stability and reliability of large language models (LLMs) as automatic evaluators for abstractive summarization. We find that while ChatGPT and GPT-4 outperform the commonly used automatic metrics, they are not ready as human replacements.
  arXiv  Detail & Related papers  (2023-05-22T14:58:13Z)
• Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks [67.86285142381644]
  Recent advances in instruction-following large language models amplify the dual-use risks for malicious purposes. Dual-use is difficult to prevent as instruction-following capabilities now enable standard attacks from computer security. We show that instruction-following LLMs can produce targeted malicious content, including hate speech and scams.
  arXiv  Detail & Related papers  (2023-02-11T15:57:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.