SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning

• URL: http://arxiv.org/abs/2204.00734v1
• Date: Sat, 2 Apr 2022 01:21:09 GMT
• Title: SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning
• Authors: Nilaksh Das, Sheng-Yun Peng, Duen Horng Chau
• Abstract summary: We study the impact of multi-task learning (MTL) on the adversarial robustness of the widely used SiamRPN tracker. Specifically, we investigate the effect of jointly learning with semantically analogous tasks of person tracking and human keypoint detection. Our empirical study with simulated as well as real-world datasets reveals that training with MTL consistently makes it harder to attack the SiamRPN tracker.
• Score: 12.245882404444881
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Person tracking using computer vision techniques has wide ranging applications such as autonomous driving, home security and sports analytics. However, the growing threat of adversarial attacks raises serious concerns regarding the security and reliability of such techniques. In this work, we study the impact of multi-task learning (MTL) on the adversarial robustness of the widely used SiamRPN tracker, in the context of person tracking. Specifically, we investigate the effect of jointly learning with semantically analogous tasks of person tracking and human keypoint detection. We conduct extensive experiments with more powerful adversarial attacks that can be physically realizable, demonstrating the practical value of our approach. Our empirical study with simulated as well as real-world datasets reveals that training with MTL consistently makes it harder to attack the SiamRPN tracker, compared to typically training only on the single task of person tracking.

Related papers

• Unveiling the Power of Self-supervision for Multi-view Multi-human Association and Tracking [22.243799150495487]
  Multi-view multi-human association and tracking (MvMHAT) is a new but important problem for multi-person scene video surveillance. We tackle this problem with a self-supervised learning aware end-to-end network. We build two new large-scale benchmarks for the network training and testing of different algorithms.
  arXiv  Detail & Related papers  (2024-01-31T06:12:28Z)
• Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios [147.16925581385576]
  We show how imitation learning combined with reinforcement learning can substantially improve the safety and reliability of driving policies. We train a policy on over 100k miles of urban driving data, and measure its effectiveness in test scenarios grouped by different levels of collision likelihood.
  arXiv  Detail & Related papers  (2022-12-21T23:59:33Z)
• Towards Robust Dataset Learning [90.2590325441068]
  We propose a principled, tri-level optimization to formulate the robust dataset learning problem. Under an abstraction model that characterizes robust vs. non-robust features, the proposed method provably learns a robust dataset.
  arXiv  Detail & Related papers  (2022-11-19T17:06:10Z)
• DIMBA: Discretely Masked Black-Box Attack in Single Object Tracking [5.672132510411465]
  adversarial attack can force a CNN-based model to produce an incorrect output by craftily manipulating human-imperceptible input. We propose a novel adversarial attack method to generate noises for single object tracking under black-box settings. Our method requires fewer queries on frames of a video to manipulate competitive or even better attack performance.
  arXiv  Detail & Related papers  (2022-07-17T00:17:40Z)
• Hear No Evil: Towards Adversarial Robustness of Automatic Speech Recognition via Multi-Task Learning [13.735883484044166]
  We investigate the impact of performing multi-task learning on the adversarial robustness of ASR models in the speech domain. Our approach shows considerable absolute improvements in adversarially targeted WER ranging from 17.25 up to 59.90. Ours is the first in-depth study that uncovers adversarial robustness gains from multi-task learning for ASR.
  arXiv  Detail & Related papers  (2022-04-05T17:40:19Z)
• Visual Adversarial Imitation Learning using Variational Models [60.69745540036375]
  Reward function specification remains a major impediment for learning behaviors through deep reinforcement learning. Visual demonstrations of desired behaviors often presents an easier and more natural way to teach agents. We develop a variational model-based adversarial imitation learning algorithm.
  arXiv  Detail & Related papers  (2021-07-16T00:15:18Z)
• Crop-Transform-Paste: Self-Supervised Learning for Visual Tracking [137.26381337333552]
  In this work, we develop the Crop-Transform-Paste operation, which is able to synthesize sufficient training data. Since the object state is known in all synthesized data, existing deep trackers can be trained in routine ways without human annotation.
  arXiv  Detail & Related papers  (2021-06-21T07:40:34Z)
• Temporally-Transferable Perturbations: Efficient, One-Shot Adversarial Attacks for Online Visual Object Trackers [81.90113217334424]
  We propose a framework to generate a single temporally transferable adversarial perturbation from the object template image only. This perturbation can then be added to every search image, which comes at virtually no cost, and still, successfully fool the tracker.
  arXiv  Detail & Related papers  (2020-12-30T15:05:53Z)
• VATLD: A Visual Analytics System to Assess, Understand and Improve Traffic Light Detection [15.36267013724161]
  We propose a visual analytics system, VATLD, to assess, understand, and improve the accuracy and robustness of traffic light detectors in autonomous driving applications. The disentangled representation learning extracts data semantics to augment human cognition with human-friendly visual summarization. We also demonstrate the effectiveness of various performance improvement strategies with our visual analytics system, VATLD, and illustrate some practical implications for safety-critical applications in autonomous driving.
  arXiv  Detail & Related papers  (2020-09-27T22:39:00Z)
• Human-in-the-Loop Methods for Data-Driven and Reinforcement Learning Systems [0.8223798883838329]
  This research investigates how to integrate human interaction modalities to the reinforcement learning loop. Results show that the reward signal that is learned based upon human interaction accelerates the rate of learning of reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2020-08-30T17:28:18Z)
• Self-supervised Object Tracking with Cycle-consistent Siamese Networks [55.040249900677225]
  We exploit an end-to-end Siamese network in a cycle-consistent self-supervised framework for object tracking. We propose to integrate a Siamese region proposal and mask regression network in our tracking framework so that a fast and more accurate tracker can be learned without the annotation of each frame.
  arXiv  Detail & Related papers  (2020-08-03T04:10:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.