The Shadow of Fraud: The Emerging Danger of AI-powered Social Engineering and its Possible Cure

• URL: http://arxiv.org/abs/2407.15912v1
• Date: Mon, 22 Jul 2024 17:37:31 GMT
• Title: The Shadow of Fraud: The Emerging Danger of AI-powered Social Engineering and its Possible Cure
• Authors: Jingru Yu, Yi Yu, Xuhong Wang, Yilun Lin, Manzhi Yang, Yu Qiao, Fei-Yue Wang,
• Abstract summary: Social engineering (SE) attacks remain a significant threat to both individuals and organizations. The advancement of Artificial Intelligence (AI) has potentially intensified these threats by enabling more personalized and convincing attacks. This survey paper categorizes SE attack mechanisms, analyzes their evolution, and explores methods for measuring these threats.
• Score: 30.431292911543103
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social engineering (SE) attacks remain a significant threat to both individuals and organizations. The advancement of Artificial Intelligence (AI), including diffusion models and large language models (LLMs), has potentially intensified these threats by enabling more personalized and convincing attacks. This survey paper categorizes SE attack mechanisms, analyzes their evolution, and explores methods for measuring these threats. It highlights the challenges in raising awareness about the risks of AI-enhanced SE attacks and offers insights into developing proactive and adaptable defense strategies. Additionally, we introduce a categorization of the evolving nature of AI-powered social engineering attacks into "3E phases": Enlarging, wherein the magnitude of attacks expands through the leverage of digital media; Enriching, introducing novel attack vectors and techniques; and Emerging, signifying the advent of novel threats and methods. Moreover, we emphasize the necessity for a robust framework to assess the risk of AI-powered SE attacks. By identifying and addressing gaps in existing research, we aim to guide future studies and encourage the development of more effective defenses against the growing threat of AI-powered social engineering.

Related papers

• Asset-centric Threat Modeling for AI-based Systems [7.696807063718328]
  This paper presents ThreatFinderAI, an approach and tool to model AI-related assets, threats, countermeasures, and quantify residual risks. To evaluate the practicality of the approach, participants were tasked to recreate a threat model developed by cybersecurity experts of an AI-based healthcare platform. Overall, the solution's usability was well-perceived and effectively supports threat identification and risk discussion.
  arXiv  Detail & Related papers  (2024-03-11T08:40:01Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• Autonomous Threat Hunting: A Future Paradigm for AI-Driven Threat Intelligence [0.0]
  Review explores the amalgamation of artificial intelligence (AI) and traditional threat intelligence methodologies. Examines the transformative influence of AI and machine learning on conventional threat intelligence practices. Case studies and evaluations highlight success stories and lessons learned by organizations adopting AI-driven threat intelligence.
  arXiv  Detail & Related papers  (2023-12-30T17:36:08Z)
• Towards more Practical Threat Models in Artificial Intelligence Security [66.67624011455423]
  Recent works have identified a gap between research and practice in artificial intelligence security. We revisit the threat models of the six most studied attacks in AI security research and match them to AI usage in practice.
  arXiv  Detail & Related papers  (2023-11-16T16:09:44Z)
• Managing extreme AI risks amid rapid progress [171.05448842016125]
  We describe risks that include large-scale social harms, malicious uses, and irreversible loss of human control over autonomous AI systems. There is a lack of consensus about how exactly such risks arise, and how to manage them. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
  arXiv  Detail & Related papers  (2023-10-26T17:59:06Z)
• Digital Deception: Generative Artificial Intelligence in Social Engineering and Phishing [7.1795069620810805]
  This paper investigates the transformative role of Generative AI in Social Engineering (SE) attacks. We use a theory of social engineering to identify three pillars where Generative AI amplifies the impact of SE attacks. Our study aims to foster a deeper understanding of the risks, human implications, and countermeasures associated with this emerging paradigm.
  arXiv  Detail & Related papers  (2023-10-15T07:55:59Z)
• Decoding the Threat Landscape : ChatGPT, FraudGPT, and WormGPT in Social Engineering Attacks [0.0]
  Generative AI models have revolutionized the field of cyberattacks, empowering malicious actors to craft convincing and personalized phishing lures. These models, ChatGPT, FraudGPT, and WormGPT, have augmented existing threats and ushered in new dimensions of risk. To counter these threats, we outline a range of strategies, including traditional security measures, AI-powered security solutions, and collaborative approaches in cybersecurity.
  arXiv  Detail & Related papers  (2023-10-09T10:31:04Z)
• On the Security Risks of Knowledge Graph Reasoning [71.64027889145261]
  We systematize the security threats to KGR according to the adversary's objectives, knowledge, and attack vectors. We present ROAR, a new class of attacks that instantiate a variety of such threats. We explore potential countermeasures against ROAR, including filtering of potentially poisoning knowledge and training with adversarially augmented queries.
  arXiv  Detail & Related papers  (2023-05-03T18:47:42Z)
• Towards Automated Classification of Attackers' TTPs by combining NLP with ML Techniques [77.34726150561087]
  We evaluate and compare different Natural Language Processing (NLP) and machine learning techniques used for security information extraction in research. Based on our investigations we propose a data processing pipeline that automatically classifies unstructured text according to attackers' tactics and techniques.
  arXiv  Detail & Related papers  (2022-07-18T09:59:21Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)
• Security and Privacy for Artificial Intelligence: Opportunities and Challenges [11.368470074697747]
  In recent years, most AI models are vulnerable to advanced and sophisticated hacking techniques. This challenge has motivated concerted research efforts into adversarial AI. We present a holistic cyber security review that demonstrates adversarial attacks against AI applications.
  arXiv  Detail & Related papers  (2021-02-09T06:06:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.