Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack

• URL: http://arxiv.org/abs/2211.11312v2
• Date: Mon, 6 May 2024 11:00:43 GMT
• Title: Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack
• Authors: Yunfeng Diao, He Wang, Tianjia Shao, Yong-Liang Yang, Kun Zhou, David Hogg, Meng Wang,
• Abstract summary: Human Activity Recognition (HAR) has been employed in a wide range of applications, e.g. self-driving cars. Recently, the robustness of skeleton-based HAR methods have been questioned due to their vulnerability to adversarial attacks. We show such threats exist, even when the attacker only has access to the input/output of the model. We propose the very first black-box adversarial attack approach in skeleton-based HAR called BASAR.
• Score: 53.032801921915436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human Activity Recognition (HAR) has been employed in a wide range of applications, e.g. self-driving cars, where safety and lives are at stake. Recently, the robustness of skeleton-based HAR methods have been questioned due to their vulnerability to adversarial attacks. However, the proposed attacks require the full-knowledge of the attacked classifier, which is overly restrictive. In this paper, we show such threats indeed exist, even when the attacker only has access to the input/output of the model. To this end, we propose the very first black-box adversarial attack approach in skeleton-based HAR called BASAR. BASAR explores the interplay between the classification boundary and the natural motion manifold. To our best knowledge, this is the first time data manifold is introduced in adversarial attacks on time series. Via BASAR, we find on-manifold adversarial samples are extremely deceitful and rather common in skeletal motions, in contrast to the common belief that adversarial samples only exist off-manifold. Through exhaustive evaluation, we show that BASAR can deliver successful attacks across classifiers, datasets, and attack modes. By attack, BASAR helps identify the potential causes of the model vulnerability and provides insights on possible improvements. Finally, to mitigate the newly identified threat, we propose a new adversarial training approach by leveraging the sophisticated distributions of on/off-manifold adversarial samples, called mixed manifold-based adversarial training (MMAT). MMAT can successfully help defend against adversarial attacks without compromising classification accuracy.

Related papers

• Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient [14.392853911242923]
  Methods for skeleton-based human activity recognition have been shown to be vulnerable to adversarial attacks. In this paper, we consider a new attack task: the attacker has no access to the victim model or the training data or labels. Specifically, we define an adversarial loss to compute a new gradient for the attack, named skeleton-motion-informed (SMI) gradient.
  arXiv  Detail & Related papers  (2023-08-10T16:34:20Z)
• Defending Black-box Skeleton-based Human Activity Classifiers [38.95979614080714]
  In this paper, we investigate skeleton-based Human Activity Recognition, which is an important type of time-series data but under-explored in defense against attacks. We name our framework Bayesian Energy-based Adversarial Training or BEAT. BEAT is straightforward but elegant, which turns vulnerable black-box classifiers into robust ones without sacrificing accuracy.
  arXiv  Detail & Related papers  (2022-03-09T13:46:10Z)
• Adversarial Robustness of Deep Reinforcement Learning based Dynamic Recommender Systems [50.758281304737444]
  We propose to explore adversarial examples and attack detection on reinforcement learning-based interactive recommendation systems. We first craft different types of adversarial examples by adding perturbations to the input and intervening on the casual factors. Then, we augment recommendation systems by detecting potential attacks with a deep learning-based classifier based on the crafted data.
  arXiv  Detail & Related papers  (2021-12-02T04:12:24Z)
• Towards A Conceptually Simple Defensive Approach for Few-shot classifiers Against Adversarial Support Samples [107.38834819682315]
  We study a conceptually simple approach to defend few-shot classifiers against adversarial attacks. We propose a simple attack-agnostic detection method, using the concept of self-similarity and filtering. Our evaluation on the miniImagenet (MI) and CUB datasets exhibit good attack detection performance.
  arXiv  Detail & Related papers  (2021-10-24T05:46:03Z)
• Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning [95.60856995067083]
  This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. We propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. Experimental results show that our detection module effectively shields the ASV by detecting adversarial samples with an accuracy of around 80%.
  arXiv  Detail & Related papers  (2021-06-01T07:10:54Z)
• ExAD: An Ensemble Approach for Explanation-based Adversarial Detection [17.455233006559734]
  We propose ExAD, a framework to detect adversarial examples using an ensemble of explanation techniques. We evaluate our approach using six state-of-the-art adversarial attacks on three image datasets.
  arXiv  Detail & Related papers  (2021-03-22T00:53:07Z)
• BASAR:Black-box Attack on Skeletal Action Recognition [32.88446909707521]
  Skeleton-based activity recognizers are vulnerable to adversarial attacks when the full-knowledge of the recognizer is accessible to the attacker. In this paper, we show that such threats do exist under black-box settings too. Through BASAR, we show that adversarial attack is not only truly a threat but also can be extremely deceitful.
  arXiv  Detail & Related papers  (2021-03-09T07:29:35Z)
• Adversarial Example Games [51.92698856933169]
  Adrial Example Games (AEG) is a framework that models the crafting of adversarial examples. AEG provides a new way to design adversarial examples by adversarially training a generator and aversa from a given hypothesis class. We demonstrate the efficacy of AEG on the MNIST and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2020-07-01T19:47:23Z)
• AdvMind: Inferring Adversary Intent of Black-Box Attacks [66.19339307119232]
  We present AdvMind, a new class of estimation models that infer the adversary intent of black-box adversarial attacks in a robust manner. On average AdvMind detects the adversary intent with over 75% accuracy after observing less than 3 query batches.
  arXiv  Detail & Related papers  (2020-06-16T22:04:31Z)
• Adversarial Detection and Correction by Matching Prediction Distributions [0.0]
  The detector almost completely neutralises powerful attacks like Carlini-Wagner or SLIDE on MNIST and Fashion-MNIST. We show that our method is still able to detect the adversarial examples in the case of a white-box attack where the attacker has full knowledge of both the model and the defence.
  arXiv  Detail & Related papers  (2020-02-21T15:45:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.