A Study of Vulnerability Repair in JavaScript Programs with Large Language Models

• URL: http://arxiv.org/abs/2403.13193v1
• Date: Tue, 19 Mar 2024 23:04:03 GMT
• Title: A Study of Vulnerability Repair in JavaScript Programs with Large Language Models
• Authors: Tan Khang Le, Saba Alimadadi, Steven Y. Ko,
• Abstract summary: Large Language Models (LLMs) have demonstrated substantial advancements across multiple domains. Our experiments on real-world software vulnerabilities show that while LLMs are promising in automatic program repair of JavaScript code, achieving a correct bug fix often requires an appropriate amount of context in the prompt.
• Score: 2.4622939109173885
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In recent years, JavaScript has become the most widely used programming language, especially in web development. However, writing secure JavaScript code is not trivial, and programmers often make mistakes that lead to security vulnerabilities in web applications. Large Language Models (LLMs) have demonstrated substantial advancements across multiple domains, and their evolving capabilities indicate their potential for automatic code generation based on a required specification, including automatic bug fixing. In this study, we explore the accuracy of LLMs, namely ChatGPT and Bard, in finding and fixing security vulnerabilities in JavaScript programs. We also investigate the impact of context in a prompt on directing LLMs to produce a correct patch of vulnerable JavaScript code. Our experiments on real-world software vulnerabilities show that while LLMs are promising in automatic program repair of JavaScript code, achieving a correct bug fix often requires an appropriate amount of context in the prompt.

Related papers

• SafePyScript: A Web-Based Solution for Machine Learning-Driven Vulnerability Detection in Python [0.0]
  We present SafePyScript, a machine learning-based web application designed specifically to identify vulnerabilities in Python source code. Despite Python's significance as a major programming language, there is currently no convenient and easy-to-use machine learning-based web application for detecting vulnerabilities in its source code.
  arXiv  Detail & Related papers  (2024-11-01T14:49:33Z)
• Aligning LLMs to Be Robust Against Prompt Injection [55.07562650579068]
  We show that alignment can be a powerful tool to make LLMs more robust against prompt injection attacks. Our method -- SecAlign -- first builds an alignment dataset by simulating prompt injection attacks. Our experiments show that SecAlign robustifies the LLM substantially with a negligible hurt on model utility.
  arXiv  Detail & Related papers  (2024-10-07T19:34:35Z)
• NAVRepair: Node-type Aware C/C++ Code Vulnerability Repair [14.152755184229374]
  NAVRepair is a novel framework that combines the node-type information extracted fromASTs with error types, specifically targeting C/C++ vulnerabilities. We achieve a 26% higher accuracy compared to an existing LLM-based C/C++ vulnerability repair method.
  arXiv  Detail & Related papers  (2024-05-08T11:58:55Z)
• CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion [117.178835165855]
  This paper introduces CodeAttack, a framework that transforms natural language inputs into code inputs. Our studies reveal a new and universal safety vulnerability of these models against code input. We find that a larger distribution gap between CodeAttack and natural language leads to weaker safety generalization.
  arXiv  Detail & Related papers  (2024-03-12T17:55:38Z)
• A Novel Approach for Automatic Program Repair using Round-Trip Translation with Large Language Models [50.86686630756207]
  Research shows that grammatical mistakes in a sentence can be corrected by translating it to another language and back. Current generative models for Automatic Program Repair (APR) are pre-trained on source code and fine-tuned for repair. This paper proposes bypassing the fine-tuning step and using Round-Trip Translation (RTT): translation of code from one programming language to another programming or natural language, and back.
  arXiv  Detail & Related papers  (2024-01-15T22:36:31Z)
• Static Semantics Reconstruction for Enhancing JavaScript-WebAssembly Multilingual Malware Detection [51.15122099046214]
  WebAssembly allows attackers to hide the malicious functionalities of JavaScript malware in cross-language interoperations. The detection of JavaScript-WebAssembly multilingual malware (JWMM) is challenging due to the complex interoperations and semantic diversity between JavaScript and WebAssembly. We present JWBinder, the first technique aimed at enhancing the static detection of JWMM.
  arXiv  Detail & Related papers  (2023-10-26T10:59:45Z)
• Can Large Language Models Find And Fix Vulnerable Software? [0.0]
  GPT-4 identified approximately four times the vulnerabilities than its counterparts. It provided viable fixes for each vulnerability, demonstrating a low rate of false positives. GPT-4's code corrections led to a 90% reduction in vulnerabilities, requiring only an 11% increase in code lines.
  arXiv  Detail & Related papers  (2023-08-20T19:33:12Z)
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [64.67495502772866]
  Large Language Models (LLMs) are increasingly being integrated into various applications. We show how attackers can override original instructions and employed controls using Prompt Injection attacks. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
  arXiv  Detail & Related papers  (2023-02-23T17:14:38Z)
• CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models [58.27254444280376]
  Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure.
  arXiv  Detail & Related papers  (2023-02-08T11:54:07Z)
• Can OpenAI Codex and Other Large Language Models Help Us Fix Security Bugs? [8.285068188878578]
  We examine the use of large language models (LLMs) for code repair. We investigate challenges in the design of prompts that coax LLMs into generating repaired versions of insecure code. Experiments show that LLMs could collectively repair 100% of our synthetically generated and hand-crafted scenarios.
  arXiv  Detail & Related papers  (2021-12-03T19:15:02Z)
• Montage: A Neural Network Language Model-Guided JavaScript Engine Fuzzer [18.908548472588976]
  We present Montage, the first NNLM-guided fuzzer for finding JS engine vulnerabilities. Montage found 37 real-world bugs, including three CVEs, in the latest JS engines.
  arXiv  Detail & Related papers  (2020-01-13T08:45:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.