When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs

• URL: http://arxiv.org/abs/2410.14569v1
• Date: Fri, 18 Oct 2024 16:16:34 GMT
• Title: When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs
• Authors: Hanna Kim, Minkyoo Song, Seung Ho Na, Seungwon Shin, Kimin Lee,
• Abstract summary: We investigate the risks associated with misuse of Large Language Models (LLMs) in cyberattacks involving personal data. Specifically, we aim to understand how potent LLM agents can be when directed to conduct cyberattacks. We examine three attack scenarios: the collection of Personally Identifiable Information (PII), the generation of impersonation posts, and the creation of spear-phishing emails.
• Score: 26.2943792874156
• License:
• Abstract: Recent advancements in Large Language Models (LLMs) have established them as agentic systems capable of planning and interacting with various tools. These LLM agents are often paired with web-based tools, enabling access to diverse sources and real-time information. Although these advancements offer significant benefits across various applications, they also increase the risk of malicious use, particularly in cyberattacks involving personal information. In this work, we investigate the risks associated with misuse of LLM agents in cyberattacks involving personal data. Specifically, we aim to understand: 1) how potent LLM agents can be when directed to conduct cyberattacks, 2) how cyberattacks are enhanced by web-based tools, and 3) how affordable and easy it becomes to launch cyberattacks using LLM agents. We examine three attack scenarios: the collection of Personally Identifiable Information (PII), the generation of impersonation posts, and the creation of spear-phishing emails. Our experiments reveal the effectiveness of LLM agents in these attacks: LLM agents achieved a precision of up to 95.9% in collecting PII, up to 93.9% of impersonation posts created by LLM agents were evaluated as authentic, and the click rate for links in spear phishing emails created by LLM agents reached up to 46.67%. Additionally, our findings underscore the limitations of existing safeguards in contemporary commercial LLMs, emphasizing the urgent need for more robust security measures to prevent the misuse of LLM agents.

Related papers

• AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents [52.13695464678006]
  This study enhances an LLM-based web agent by simply refining its observation and action space. AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively.
  arXiv  Detail & Related papers  (2024-10-17T17:50:38Z)
• AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents [84.96249955105777]
  LLM agents may pose a greater risk if misused, but their robustness remains underexplored. We propose a new benchmark called AgentHarm to facilitate research on LLM agent misuse. We find leading LLMs are surprisingly compliant with malicious agent requests without jailbreaking.
  arXiv  Detail & Related papers  (2024-10-11T17:39:22Z)
• Aligning LLMs to Be Robust Against Prompt Injection [55.07562650579068]
  We show that alignment can be a powerful tool to make LLMs more robust against prompt injection attacks. Our method -- SecAlign -- first builds an alignment dataset by simulating prompt injection attacks. Our experiments show that SecAlign robustifies the LLM substantially with a negligible hurt on model utility.
  arXiv  Detail & Related papers  (2024-10-07T19:34:35Z)
• MaPPing Your Model: Assessing the Impact of Adversarial Attacks on LLM-based Programming Assistants [14.947665219536708]
  We introduce the Malicious Programming Prompt (MaPP) attack, in which an attacker adds a small amount of text to a prompt for a programming task. We show that our prompt strategy can cause an LLM to add vulnerabilities while continuing to write otherwise correct code.
  arXiv  Detail & Related papers  (2024-07-12T22:30:35Z)
• BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents [26.057916556444333]
  We show that such methods are vulnerable to our proposed backdoor attacks named BadAgent. Our proposed attack methods are extremely robust even after fine-tuning on trustworthy data.
  arXiv  Detail & Related papers  (2024-06-05T07:14:28Z)
• Prompt Leakage effect and defense strategies for multi-turn LLM interactions [95.33778028192593]
  Leakage of system prompts may compromise intellectual property and act as adversarial reconnaissance for an attacker. We design a unique threat model which leverages the LLM sycophancy effect and elevates the average attack success rate (ASR) from 17.7% to 86.2% in a multi-turn setting. We measure the mitigation effect of 7 black-box defense strategies, along with finetuning an open-source model to defend against leakage attempts.
  arXiv  Detail & Related papers  (2024-04-24T23:39:58Z)
• The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative [55.08395463562242]
  Multimodal Large Language Models (MLLMs) are constantly defining the new boundary of Artificial General Intelligence (AGI) Our paper explores a novel vulnerability in MLLM societies - the indirect propagation of malicious content.
  arXiv  Detail & Related papers  (2024-02-20T23:08:21Z)
• Detecting Phishing Sites Using ChatGPT [2.3999111269325266]
  We propose a novel system called ChatPhishDetector that utilizes Large Language Models (LLMs) to detect phishing sites. Our system involves leveraging a web crawler to gather information from websites, generating prompts for LLMs based on the crawled data, and then retrieving the detection results from the responses generated by the LLMs. The experimental results using GPT-4V demonstrated outstanding performance, with a precision of 98.7% and a recall of 99.6%, outperforming the detection results of other LLMs and existing systems.
  arXiv  Detail & Related papers  (2023-06-09T11:30:08Z)
• Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks [67.86285142381644]
  Recent advances in instruction-following large language models amplify the dual-use risks for malicious purposes. Dual-use is difficult to prevent as instruction-following capabilities now enable standard attacks from computer security. We show that instruction-following LLMs can produce targeted malicious content, including hate speech and scams.
  arXiv  Detail & Related papers  (2023-02-11T15:57:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.