Prompt-Guided Zero-Shot Anomaly Action Recognition using Pretrained Deep
Skeleton Features
- URL: http://arxiv.org/abs/2303.15167v1
- Date: Mon, 27 Mar 2023 12:59:33 GMT
- Title: Prompt-Guided Zero-Shot Anomaly Action Recognition using Pretrained Deep
Skeleton Features
- Authors: Fumiaki Sato, Ryo Hachiuma, Taiki Sekii
- Abstract summary: Unsupervised anomaly action recognition identifies video-level abnormal-human-behavior events in an unsupervised manner without abnormal samples.
We present a unified, user prompt-guided zero-shot learning framework using a target domain-independent skeleton feature extractor.
We incorporate a similarity score between the user prompt embeddings and skeleton features aligned in the common space into the anomaly score, which indirectly supplements normal actions.
- Score: 3.255030588361124
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: This study investigates unsupervised anomaly action recognition, which
identifies video-level abnormal-human-behavior events in an unsupervised manner
without abnormal samples, and simultaneously addresses three limitations in the
conventional skeleton-based approaches: target domain-dependent DNN training,
robustness against skeleton errors, and a lack of normal samples. We present a
unified, user prompt-guided zero-shot learning framework using a target
domain-independent skeleton feature extractor, which is pretrained on a
large-scale action recognition dataset. Particularly, during the training phase
using normal samples, the method models the distribution of skeleton features
of the normal actions while freezing the weights of the DNNs and estimates the
anomaly score using this distribution in the inference phase. Additionally, to
increase robustness against skeleton errors, we introduce a DNN architecture
inspired by a point cloud deep learning paradigm, which sparsely propagates the
features between joints. Furthermore, to prevent the unobserved normal actions
from being misidentified as abnormal actions, we incorporate a similarity score
between the user prompt embeddings and skeleton features aligned in the common
space into the anomaly score, which indirectly supplements normal actions. On
two publicly available datasets, we conduct experiments to test the
effectiveness of the proposed method with respect to abovementioned
limitations.
Related papers
- Frequency-Guided Multi-Level Human Action Anomaly Detection with Normalizing Flows [13.77542608443034]
We introduce the task of human action anomaly detection (HAAD), which aims to identify anomalous motions in an unsupervised manner.
Compared to prior human-related anomaly detection tasks which primarily focus on unusual events from videos, HAAD involves the learning of specific action labels to recognize semantically anomalous human behaviors.
arXiv Detail & Related papers (2024-04-26T12:56:16Z) - Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image.
In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting.
Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
arXiv Detail & Related papers (2023-11-27T13:14:06Z) - Unveiling the Hidden Realm: Self-supervised Skeleton-based Action
Recognition in Occluded Environments [41.664437160034176]
We propose a simple and effective method to empower robots with the capacity to address occlusion.
We first pre-train using occluded skeleton sequences, then use k-means clustering (KMeans) on sequence embeddings to group semantically similar samples.
We then employ K-nearest-neighbor (KNN) to fill in missing skeleton data based on the closest sample neighbors.
arXiv Detail & Related papers (2023-09-21T12:51:11Z) - An Iterative Method for Unsupervised Robust Anomaly Detection Under Data
Contamination [24.74938110451834]
Most deep anomaly detection models are based on learning normality from datasets.
In practice, the normality assumption is often violated due to the nature of real data distributions.
We propose a learning framework to reduce this gap and achieve better normality representation.
arXiv Detail & Related papers (2023-09-18T02:36:19Z) - On the Universal Adversarial Perturbations for Efficient Data-free
Adversarial Detection [55.73320979733527]
We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs.
Experimental results show that our method achieves competitive detection performance on various text classification tasks.
arXiv Detail & Related papers (2023-06-27T02:54:07Z) - Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold.
We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples.
We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
arXiv Detail & Related papers (2022-06-23T14:16:30Z) - Object-centric and memory-guided normality reconstruction for video
anomaly detection [56.64792194894702]
This paper addresses anomaly detection problem for videosurveillance.
Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy.
Our model learns object-centric normal patterns without seeing anomalous samples during training.
arXiv Detail & Related papers (2022-03-07T19:28:39Z) - SLA$^2$P: Self-supervised Anomaly Detection with Adversarial
Perturbation [77.71161225100927]
Anomaly detection is a fundamental yet challenging problem in machine learning.
We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
arXiv Detail & Related papers (2021-11-25T03:53:43Z) - Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
We introduce a novel weakly-supervised anomaly detection framework to train detection models.
The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability.
Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
arXiv Detail & Related papers (2021-08-01T14:33:17Z) - Learning Memory-guided Normality for Anomaly Detection [33.77435699029528]
We present an unsupervised learning approach to anomaly detection that considers the diversity of normal patterns explicitly.
We also present novel feature compactness and separateness losses to train the memory, boosting the discriminative power of both memory items and deeply learned features from normal data.
arXiv Detail & Related papers (2020-03-30T05:30:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.