Learn Suspected Anomalies from Event Prompts for Video Anomaly Detection
- URL: http://arxiv.org/abs/2403.01169v1
- Date: Sat, 2 Mar 2024 10:42:47 GMT
- Title: Learn Suspected Anomalies from Event Prompts for Video Anomaly Detection
- Authors: Chenchen Tao, Chong Wang, Yuexian Zou, Xiaohao Peng, Jiafei Wu and
Jiangbo Qian
- Abstract summary: A novel framework is proposed to guide the learning of suspected anomalies from event prompts.
It enables a new multi-prompt learning process to constrain the visual-semantic features across all videos.
Our proposed model outperforms most state-of-the-art methods in terms of AP or AUC.
- Score: 49.91075101563298
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Most models for weakly supervised video anomaly detection (WS-VAD) rely on
multiple instance learning, aiming to distinguish normal and abnormal snippets
without specifying the type of anomaly. The ambiguous nature of anomaly
definitions across contexts introduces bias in detecting abnormal and normal
snippets within the abnormal bag. Taking the first step to show the model why
it is anomalous, a novel framework is proposed to guide the learning of
suspected anomalies from event prompts. Given a textual prompt dictionary of
potential anomaly events and the captions generated from anomaly videos, the
semantic anomaly similarity between them could be calculated to identify the
suspected anomalous events for each video snippet. It enables a new
multi-prompt learning process to constrain the visual-semantic features across
all videos, as well as provides a new way to label pseudo anomalies for
self-training. To demonstrate effectiveness, comprehensive experiments and
detailed ablation studies are conducted on four datasets, namely XD-Violence,
UCF-Crime, TAD, and ShanghaiTech. Our proposed model outperforms most
state-of-the-art methods in terms of AP or AUC (82.6\%, 87.7\%, 93.1\%, and
97.4\%). Furthermore, it shows promising performance in open-set and
cross-dataset cases.
Related papers
- Open-Vocabulary Video Anomaly Detection [57.552523669351636]
Video anomaly detection (VAD) with weak supervision has achieved remarkable performance in utilizing video-level labels to discriminate whether a video frame is normal or abnormal.
Recent studies attempt to tackle a more realistic setting, open-set VAD, which aims to detect unseen anomalies given seen anomalies and normal videos.
This paper takes a step further and explores open-vocabulary video anomaly detection (OVVAD), in which we aim to leverage pre-trained large models to detect and categorize seen and unseen anomalies.
arXiv Detail & Related papers (2023-11-13T02:54:17Z) - Catching Both Gray and Black Swans: Open-set Supervised Anomaly
Detection [90.32910087103744]
A few labeled anomaly examples are often available in many real-world applications.
These anomaly examples provide valuable knowledge about the application-specific abnormality.
Those anomalies seen during training often do not illustrate every possible class of anomaly.
This paper tackles open-set supervised anomaly detection.
arXiv Detail & Related papers (2022-03-28T05:21:37Z) - Anomaly Crossing: A New Method for Video Anomaly Detection as
Cross-domain Few-shot Learning [32.0713939637202]
Video anomaly detection aims to identify abnormal events that occurred in videos.
Most previous approaches learn only from normal videos using unsupervised or semi-supervised methods.
We propose a new learning paradigm by making full use of both normal and abnormal videos for video anomaly detection.
arXiv Detail & Related papers (2021-12-12T20:49:38Z) - UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection [103.06327681038304]
We propose a supervised open-set benchmark composed of multiple virtual scenes for video anomaly detection.
Unlike existing data sets, we introduce abnormal events annotated at the pixel level at training time.
We show that UBnormal can enhance the performance of a state-of-the-art anomaly detection framework.
arXiv Detail & Related papers (2021-11-16T17:28:46Z) - Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
We introduce a novel weakly-supervised anomaly detection framework to train detection models.
The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability.
Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
arXiv Detail & Related papers (2021-08-01T14:33:17Z) - Unsupervised Video Anomaly Detection via Normalizing Flows with Implicit
Latent Features [8.407188666535506]
Most existing methods use an autoencoder to learn to reconstruct normal videos.
We propose an implicit two-path AE (ITAE), a structure in which two encoders implicitly model appearance and motion features.
For the complex distribution of normal scenes, we suggest normal density estimation of ITAE features.
NF models intensify ITAE performance by learning normality through implicitly learned features.
arXiv Detail & Related papers (2020-10-15T05:02:02Z) - A Background-Agnostic Framework with Adversarial Training for Abnormal
Event Detection in Video [120.18562044084678]
Abnormal event detection in video is a complex computer vision problem that has attracted significant attention in recent years.
We propose a background-agnostic framework that learns from training videos containing only normal events.
arXiv Detail & Related papers (2020-08-27T18:39:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.