SmartLLM: Smart Contract Auditing using Custom Generative AI

• URL: http://arxiv.org/abs/2502.13167v1
• Date: Mon, 17 Feb 2025 06:22:05 GMT
• Title: SmartLLM: Smart Contract Auditing using Custom Generative AI
• Authors: Jun Kevin, Pujianto Yugopuspito,
• Abstract summary: This paper introduces SmartLLM, a novel approach leveraging fine-tuned LLaMA 3.1 models with Retrieval-Augmented Generation (RAG)<n>By integrating domain-specific knowledge from ERC standards, SmartLLM achieves superior performance compared to static analysis tools like Mythril and Slither.<n> Experimental results demonstrate a perfect recall of 100% and an accuracy score of 70%, highlighting the model's robustness in identifying vulnerabilities.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Smart contracts are essential to decentralized finance (DeFi) and blockchain ecosystems but are increasingly vulnerable to exploits due to coding errors and complex attack vectors. Traditional static analysis tools and existing vulnerability detection methods often fail to address these challenges comprehensively, leading to high false-positive rates and an inability to detect dynamic vulnerabilities. This paper introduces SmartLLM, a novel approach leveraging fine-tuned LLaMA 3.1 models with Retrieval-Augmented Generation (RAG) to enhance the accuracy and efficiency of smart contract auditing. By integrating domain-specific knowledge from ERC standards and employing advanced techniques such as QLoRA for efficient fine-tuning, SmartLLM achieves superior performance compared to static analysis tools like Mythril and Slither, as well as zero-shot large language model (LLM) prompting methods such as GPT-3.5 and GPT-4. Experimental results demonstrate a perfect recall of 100% and an accuracy score of 70%, highlighting the model's robustness in identifying vulnerabilities, including reentrancy and access control issues. This research advances smart contract security by offering a scalable and effective auditing solution, supporting the secure adoption of decentralized applications.

Related papers

• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage. We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility. Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• Enhancing Smart Contract Vulnerability Detection in DApps Leveraging Fine-Tuned LLM [0.7018579932647147]
  Decentralized applications (DApps) face significant security risks due to vulnerabilities in smart contracts. This paper proposes a novel approach leveraging fine-tuned Large Language Models (LLMs) to enhance smart contract vulnerability detection.
  arXiv  Detail & Related papers  (2025-04-07T12:32:14Z)
• SMART: Self-Aware Agent for Tool Overuse Mitigation [58.748554080273585]
  Current Large Language Model (LLM) agents demonstrate strong reasoning and tool use capabilities, but often lack self-awareness.<n>This imbalance leads to Tool Overuse, where models unnecessarily rely on external tools for tasks with parametric knowledge.<n>We introduce SMART (Strategic Model-Aware Reasoning with Tools), a paradigm that enhances an agent's self-awareness to optimize task handling and reduce tool overuse.
  arXiv  Detail & Related papers  (2025-02-17T04:50:37Z)
• The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility? [54.18519360412294]
  Large Language Models (LLMs) must balance between rejecting harmful requests for safety and accommodating legitimate ones for utility. This paper presents a Direct Preference Optimization (DPO) based alignment framework that achieves better overall performance. We analyze experimental results obtained from testing DeepSeek-R1 on our benchmark and reveal the critical ethical concerns raised by this highly acclaimed model.
  arXiv  Detail & Related papers  (2025-01-20T06:35:01Z)
• SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework [0.0]
  This paper introduces SmartLLMSentry, a novel framework that leverages large language models (LLMs) to advance smart contract vulnerability detection.<n>We created a specialized dataset of five randomly selected vulnerabilities for model training and evaluation.<n>Our results show an exact match accuracy of 91.1% with sufficient data, although GPT-4 demonstrated reduced performance compared to GPT-3 in rule generation.
  arXiv  Detail & Related papers  (2024-11-28T16:02:01Z)
• Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities [63.603861880022954]
  We introduce ADV-LLM, an iterative self-tuning process that crafts adversarial LLMs with enhanced jailbreak ability. Our framework significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs. It exhibits strong attack transferability to closed-source models, achieving 99% ASR on GPT-3.5 and 49% ASR on GPT-4, despite being optimized solely on Llama3.
  arXiv  Detail & Related papers  (2024-10-24T06:36:12Z)
• FTSmartAudit: A Knowledge Distillation-Enhanced Framework for Automated Smart Contract Auditing Using Fine-Tuned LLMs [17.76505488643214]
  This paper investigates the feasibility of using smaller, fine-tuned models to achieve comparable or even superior results in smart contract auditing.<n>We introduce the FTSmartAudit framework, which is designed to develop cost-effective, specialized models for smart contract auditing.<n>Our contributions include: (1) a single-task learning framework that streamlines data preparation, training, evaluation, and continuous learning; (2) a robust dataset generation method utilizing domain-special knowledge distillation to produce high-quality datasets from advanced models like GPT-4o; and (3) an adaptive learning strategy to maintain model accuracy and robustness.
  arXiv  Detail & Related papers  (2024-10-17T09:09:09Z)
• LLM-SmartAudit: Advanced Smart Contract Vulnerability Detection [3.1409266162146467]
  This paper introduces LLM-SmartAudit, a novel framework to detect and analyze vulnerabilities in smart contracts. Using a multi-agent conversational approach, LLM-SmartAudit employs a collaborative system with specialized agents to enhance the audit process. Our framework can detect complex logic vulnerabilities that traditional tools have previously overlooked.
  arXiv  Detail & Related papers  (2024-10-12T06:24:21Z)
• Vulnerability Detection in Smart Contracts: A Comprehensive Survey [10.076412566428756]
  This study examines the potential of machine learning techniques to improve the detection and mitigation of vulnerabilities in smart contracts. We analysed 88 articles published between 2018 and 2023 from the following databases: IEEE, ACM, ScienceDirect, Scopus, and Google Scholar. The findings reveal that classical machine learning techniques, including KNN, RF, DT, XG-Boost, and SVM, outperform static tools in vulnerability detection.
  arXiv  Detail & Related papers  (2024-07-08T11:51:15Z)
• Efficiently Detecting Reentrancy Vulnerabilities in Complex Smart Contracts [35.26195628798847]
  Existing vulnerability detection tools perform poorly in terms of efficiency and successful detection rates for vulnerabilities in complex contracts. SliSE provides a robust and efficient method for detection of Reentrancy vulnerabilities for complex contracts.
  arXiv  Detail & Related papers  (2024-03-17T16:08:30Z)
• ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models [65.79770974145983]
  ASSERT, Automated Safety Scenario Red Teaming, consists of three methods -- semantically aligned augmentation, target bootstrapping, and adversarial knowledge injection. We partition our prompts into four safety domains for a fine-grained analysis of how the domain affects model performance. We find statistically significant performance differences of up to 11% in absolute classification accuracy among semantically related scenarios and error rates of up to 19% absolute error in zero-shot adversarial settings.
  arXiv  Detail & Related papers  (2023-10-14T17:10:28Z)
• Enhancing Smart Contract Security Analysis with Execution Property Graphs [48.31617821205042]
  We introduce Clue, a dynamic analysis framework specifically designed for a runtime virtual machine. Clue captures critical information during contract executions, employing a novel graph-based representation, the Execution Property Graph. evaluation results reveal Clue's superior performance with high true positive rates and low false positive rates, outperforming state-of-the-art tools.
  arXiv  Detail & Related papers  (2023-05-23T13:16:42Z)
• ESCORT: Ethereum Smart COntRacTs Vulnerability Detection using Deep Neural Network and Transfer Learning [80.85273827468063]
  Existing machine learning-based vulnerability detection methods are limited and only inspect whether the smart contract is vulnerable. We propose ESCORT, the first Deep Neural Network (DNN)-based vulnerability detection framework for smart contracts. We show that ESCORT achieves an average F1-score of 95% on six vulnerability types and the detection time is 0.02 seconds per contract.
  arXiv  Detail & Related papers  (2021-03-23T15:04:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.