A Case Study on the Use of Representativeness Bias as a Defense Against Adversarial Cyber Threats

• URL: http://arxiv.org/abs/2504.20245v1
• Date: Mon, 28 Apr 2025 20:30:28 GMT
• Title: A Case Study on the Use of Representativeness Bias as a Defense Against Adversarial Cyber Threats
• Authors: Briland Hitaj, Grit Denker, Laura Tinnel, Michael McAnally, Bruce DeBruhl, Nathan Bunting, Alex Fafard, Daniel Aaron, Richard D. Roberts, Joshua Lawson, Greg McCain, Dylan Starink,
• Abstract summary: This paper takes a first step towards psychology-informed, active defense strategies.<n>Using capture-the-flag events, we create realistic challenges that tap into a particular cognitive bias: representativeness.<n>This study finds that this bias can be triggered to thwart hacking attempts and divert hackers into non-vulnerable attack paths.
• Score: 1.74585489563148
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cyberspace is an ever-evolving battleground involving adversaries seeking to circumvent existing safeguards and defenders aiming to stay one step ahead by predicting and mitigating the next threat. Existing mitigation strategies have focused primarily on solutions that consider software or hardware aspects, often ignoring the human factor. This paper takes a first step towards psychology-informed, active defense strategies, where we target biases that human beings are susceptible to under conditions of uncertainty. Using capture-the-flag events, we create realistic challenges that tap into a particular cognitive bias: representativeness. This study finds that this bias can be triggered to thwart hacking attempts and divert hackers into non-vulnerable attack paths. Participants were exposed to two different challenges designed to exploit representativeness biases. One of the representativeness challenges significantly thwarted attackers away from vulnerable attack vectors and onto non-vulnerable paths, signifying an effective bias-based defense mechanism. This work paves the way towards cyber defense strategies that leverage additional human biases to thwart future, sophisticated adversarial attacks.

Related papers

• Guarding Against Malicious Biased Threats (GAMBiT): Experimental Design of Cognitive Sensors and Triggers with Behavioral Impact Analysis [17.809804870177192]
  GAMBiT embeds insights from cognitive science into cyber environments through cognitive triggers.<n>GAMBiT establishes a new paradigm in which the attacker's mind becomes part of the battlefield.
  arXiv  Detail & Related papers  (2025-11-27T02:18:03Z)
• Debiased Dual-Invariant Defense for Adversarially Robust Person Re-Identification [52.63017280231648]
  Person re-identification (ReID) is a fundamental task in many real-world applications such as pedestrian trajectory tracking.<n>Person ReID models are highly susceptible to adversarial attacks, where imperceptible perturbations to pedestrian images can cause entirely incorrect predictions.<n>We propose a dual-invariant defense framework composed of two main phases.
  arXiv  Detail & Related papers  (2025-11-13T03:56:40Z)
• Towards Proactive Defense Against Cyber Cognitive Attacks [3.357544650969485]
  Cyber cognitive attacks leverage disruptive innovations (DIs) to exploit psychological biases and manipulate decision-making processes.<n>New technologies, such as AI-driven disinformation and synthetic media, have accelerated the scale and sophistication of these threats.<n>We introduce a novel predictive methodology for forecasting the emergence of DIs and their malicious uses in cognitive attacks.
  arXiv  Detail & Related papers  (2025-10-17T16:25:47Z)
• The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections [74.60337113759313]
  Current defenses against jailbreaks and prompt injections are typically evaluated against a static set of harmful attack strings.<n>We argue that this evaluation process is flawed. Instead, we should evaluate defenses against adaptive attackers who explicitly modify their attack strategy to counter a defense's design.
  arXiv  Detail & Related papers  (2025-10-10T05:51:04Z)
• Quantifying Loss Aversion in Cyber Adversaries via LLM Analysis [2.798191832420146]
  IARPA's ReSCIND program seeks to infer, defend against, and exploit attacker cognitive traits.<n>In this paper, we present a novel methodology that leverages large language models (LLMs) to extract quantifiable insights into the cognitive bias of loss aversion from hacker behavior.
  arXiv  Detail & Related papers  (2025-08-18T05:51:30Z)
• Modeling Behavioral Preferences of Cyber Adversaries Using Inverse Reinforcement Learning [4.5456862813416565]
  This paper presents a holistic approach to attacker preference modeling from system-level audit logs using inverse reinforcement learning (IRL)<n>We learn the behavioral preferences of cyber adversaries from forensics data on their tools and techniques.<n>Our results demonstrate for the first time that low-level forensics data can automatically reveal an adversary's subjective preferences.
  arXiv  Detail & Related papers  (2025-05-02T18:20:14Z)
• Fast Preemption: Forward-Backward Cascade Learning for Efficient and Transferable Preemptive Adversarial Defense [13.252842556505174]
  Fast Preemption is a novel preemptive adversarial defense that overcomes efficiency challenges while achieving state-of-the-art robustness and transferability.<n>Executing in just three iterations, Fast Preemption outperforms existing training-time, test-time, and preemptive defenses.
  arXiv  Detail & Related papers  (2024-07-22T10:23:44Z)
• Towards in-situ Psychological Profiling of Cybercriminals Using Dynamically Generated Deception Environments [0.0]
  Cybercrime is estimated to cost the global economy almost $10 trillion annually. Traditional perimeter security approach to cyber defence has so far proved inadequate to combat the growing threat of cybercrime. Deceptive techniques aim to mislead attackers, diverting them from critical assets whilst simultaneously gathering cyber threat intelligence on the threat actor. This article presents a proof-of-concept system that has been developed to capture the profile of an attacker in-situ, during a simulated cyber-attack in real time.
  arXiv  Detail & Related papers  (2024-05-19T09:48:59Z)
• On the Difficulty of Defending Contrastive Learning against Backdoor Attacks [58.824074124014224]
  We show how contrastive backdoor attacks operate through distinctive mechanisms. Our findings highlight the need for defenses tailored to the specificities of contrastive backdoor attacks.
  arXiv  Detail & Related papers  (2023-12-14T15:54:52Z)
• Game of Travesty: Decoy-based Psychological Cyber Deception for Proactive Human Agents [13.47548023934913]
  In this work, we adopt a signaling game framework between a defender and a human agent to develop a cyber defensive deception protocol. The proposed framework leads to fundamental theories in designing more effective signaling schemes.
  arXiv  Detail & Related papers  (2023-09-23T15:27:26Z)
• Learning to Defend by Attacking (and Vice-Versa): Transfer of Learning in Cybersecurity Games [1.14219428942199]
  We present a novel model of human decision-making inspired by the cognitive faculties of Instance-Based Learning Theory, Theory of Mind, and Transfer of Learning. This model functions by learning from both roles in a security scenario: defender and attacker, and by making predictions of the opponent's beliefs, intentions, and actions. Results from simulation experiments demonstrate the potential usefulness of cognitively inspired models of agents trained in attack and defense roles.
  arXiv  Detail & Related papers  (2023-06-03T17:51:04Z)
• On Almost-Sure Intention Deception Planning that Exploits Imperfect Observers [24.11353445650682]
  Intention deception involves computing a strategy which deceives the opponent into a wrong belief about the agent's intention or objective. This paper studies a class of probabilistic planning problems with intention deception and investigates how a defender's limited sensing modality can be exploited.
  arXiv  Detail & Related papers  (2022-09-01T16:38:03Z)
• On the Effectiveness of Adversarial Training against Backdoor Attacks [111.8963365326168]
  A backdoored model always predicts a target class in the presence of a predefined trigger pattern. In general, adversarial training is believed to defend against backdoor attacks. We propose a hybrid strategy which provides satisfactory robustness across different backdoor attacks.
  arXiv  Detail & Related papers  (2022-02-22T02:24:46Z)
• Unrestricted Adversarial Attacks on ImageNet Competition [70.8952435964555]
  Unrestricted adversarial attack is popular and practical direction but has not been studied thoroughly. We organize this competition with the purpose of exploring more effective unrestricted adversarial attack algorithm.
  arXiv  Detail & Related papers  (2021-10-17T04:27:15Z)
• Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain [58.30296637276011]
  This paper summarizes the latest research on adversarial attacks against security solutions based on machine learning techniques. It is the first to discuss the unique challenges of implementing end-to-end adversarial attacks in the cyber security domain.
  arXiv  Detail & Related papers  (2020-07-05T18:22:40Z)
• AdvMind: Inferring Adversary Intent of Black-Box Attacks [66.19339307119232]
  We present AdvMind, a new class of estimation models that infer the adversary intent of black-box adversarial attacks in a robust manner. On average AdvMind detects the adversary intent with over 75% accuracy after observing less than 3 query batches.
  arXiv  Detail & Related papers  (2020-06-16T22:04:31Z)
• Deflecting Adversarial Attacks [94.85315681223702]
  We present a new approach towards ending this cycle where we "deflect" adversarial attacks by causing the attacker to produce an input that resembles the attack's target class. We first propose a stronger defense based on Capsule Networks that combines three detection mechanisms to achieve state-of-the-art detection performance.
  arXiv  Detail & Related papers  (2020-02-18T06:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.