FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks

• URL: http://arxiv.org/abs/2601.22485v1
• Date: Fri, 30 Jan 2026 02:48:52 GMT
• Title: FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks
• Authors: Naen Xu, Jinghuai Zhang, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Zhaoxiang Wang, Shouling Ji,
• Abstract summary: Large language models (LLMs) have been widely integrated into critical automated processes.<n>They are susceptible to manipulation by fraudulent information, which can lead to harmful outcomes.<n>We introduce FraudShield, a framework designed to protect LLMs from fraudulent content by leveraging a comprehensive analysis of fraud tactics.
• Score: 42.60338754083519
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have been widely integrated into critical automated workflows, including contract review and job application processes. However, LLMs are susceptible to manipulation by fraudulent information, which can lead to harmful outcomes. Although advanced defense methods have been developed to address this issue, they often exhibit limitations in effectiveness, interpretability, and generalizability, particularly when applied to LLM-based applications. To address these challenges, we introduce FraudShield, a novel framework designed to protect LLMs from fraudulent content by leveraging a comprehensive analysis of fraud tactics. Specifically, FraudShield constructs and refines a fraud tactic-keyword knowledge graph to capture high-confidence associations between suspicious text and fraud techniques. The structured knowledge graph augments the original input by highlighting keywords and providing supporting evidence, guiding the LLM toward more secure responses. Extensive experiments show that FraudShield consistently outperforms state-of-the-art defenses across four mainstream LLMs and five representative fraud types, while also offering interpretable clues for the model's generations.

Related papers

• Friend or Foe: How LLMs' Safety Mind Gets Fooled by Intent Shift Attack [53.34204977366491]
  Large language models (LLMs) remain vulnerable to jailbreaking attacks despite their impressive capabilities.<n>In this paper, we introduce ISA (Intent Shift Attack), which obfuscates LLMs about the intent of the attacks.<n>Our approach only needs minimal edits to the original request, and yields natural, human-readable, and seemingly harmless prompts.
  arXiv  Detail & Related papers  (2025-11-01T13:44:42Z)
• Harmful Prompt Laundering: Jailbreaking LLMs with Abductive Styles and Symbolic Encoding [19.92751862281067]
  Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but their potential misuse for harmful purposes remains a significant concern.<n>We propose textbfHarmful textbfPrompt textbfLaundering (HaPLa), a novel and broadly applicable jailbreaking technique that requires only black-box access to target models.
  arXiv  Detail & Related papers  (2025-09-13T18:07:56Z)
• Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Models [61.916827858666906]
  Large Language Models (LLMs) are increasingly being integrated into services such as ChatGPT to provide responses to user queries.<n>This paper proposes a method called Token Highlighter to inspect and mitigate the potential jailbreak threats in the user query.
  arXiv  Detail & Related papers  (2024-12-24T05:10:02Z)
• Emerging Security Challenges of Large Language Models [6.151633954305939]
  Large language models (LLMs) have achieved record adoption in a short period of time across many different sectors.<n>They are open-ended models trained on diverse data without being tailored for specific downstream tasks.<n>Traditional Machine Learning (ML) models are vulnerable to adversarial attacks.
  arXiv  Detail & Related papers  (2024-12-23T14:36:37Z)
• Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM [53.79753074854936]
  Large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks.<n>This vulnerability poses significant risks to real-world applications.<n>We propose a novel defensive paradigm called GuidelineLLM.
  arXiv  Detail & Related papers  (2024-12-10T12:42:33Z)
• Global Challenge for Safe and Secure LLMs Track 1 [57.08717321907755]
  The Global Challenge for Safe and Secure Large Language Models (LLMs) is a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) This paper introduces the Global Challenge for Safe and Secure Large Language Models (LLMs), a pioneering initiative organized by AI Singapore (AISG) and the CyberSG R&D Programme Office (CRPO) to foster the development of advanced defense mechanisms against automated jailbreaking attacks.
  arXiv  Detail & Related papers  (2024-11-21T08:20:31Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• Genshin: General Shield for Natural Language Processing with Large Language Models [6.228210545695852]
  Large language models (LLMs) have been trending recently, demonstrating considerable advancement and generalizability power in countless domains. LLMs create an even bigger black box exacerbating opacity, with interpretability limited to few approaches. We propose a novel cascading framework called Genshin that combines the generalizability of the LLM, the discrimination of the median model, and the interpretability of the simple model.
  arXiv  Detail & Related papers  (2024-05-29T04:04:05Z)
• Detecting Scams Using Large Language Models [19.7220607313348]
  Large Language Models (LLMs) have gained prominence in various applications, including security. This paper explores the utility of LLMs in scam detection, a critical aspect of cybersecurity. We propose a novel use case for LLMs to identify scams, such as phishing, advance fee fraud, and romance scams.
  arXiv  Detail & Related papers  (2024-02-05T16:13:54Z)
• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly [21.536079040559517]
  Large Language Models (LLMs) have revolutionized natural language understanding and generation. This paper explores the intersection of LLMs with security and privacy.
  arXiv  Detail & Related papers  (2023-12-04T16:25:18Z)
• Spear Phishing With Large Language Models [3.2634122554914002]
  This study explores how large language models (LLMs) can be used for spear phishing. I create unique spear phishing messages for over 600 British Members of Parliament using OpenAI's GPT-3.5 and GPT-4 models. My findings provide some evidence that these messages are not only realistic but also cost-effective, with each email costing only a fraction of a cent to generate.
  arXiv  Detail & Related papers  (2023-05-11T16:55:19Z)
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [64.67495502772866]
  Large Language Models (LLMs) are increasingly being integrated into various applications. We show how attackers can override original instructions and employed controls using Prompt Injection attacks. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
  arXiv  Detail & Related papers  (2023-02-23T17:14:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.