Frequency-Guided Diffusion Model with Perturbation Training for Skeleton-Based Video Anomaly Detection

• URL: http://arxiv.org/abs/2412.03044v1
• Date: Wed, 04 Dec 2024 05:43:53 GMT
• Title: Frequency-Guided Diffusion Model with Perturbation Training for Skeleton-Based Video Anomaly Detection
• Authors: Xiaofeng Tan, Hongsong Wang, Xin Geng,
• Abstract summary: Video anomaly detection is an essential yet challenging open-set task in computer vision. Existing reconstruction-based methods encounter challenges in two main aspects: (1) limited model robustness for open-set scenarios, (2) and an overemphasis on, but restricted capacity for, detailed motion reconstruction. We propose a novel frequency-guided diffusion model with perturbation training, which enhances the model robustness by perturbation training.
• Score: 41.3349755014379
• License:
• Abstract: Video anomaly detection is an essential yet challenging open-set task in computer vision, often addressed by leveraging reconstruction as a proxy task. However, existing reconstruction-based methods encounter challenges in two main aspects: (1) limited model robustness for open-set scenarios, (2) and an overemphasis on, but restricted capacity for, detailed motion reconstruction. To this end, we propose a novel frequency-guided diffusion model with perturbation training, which enhances the model robustness by perturbation training and emphasizes the principal motion components guided by motion frequencies. Specifically, we first use a trainable generator to produce perturbative samples for perturbation training of the diffusion model. During the perturbation training phase, the model robustness is enhanced and the domain of the reconstructed model is broadened by training against this generator. Subsequently, perturbative samples are introduced for inference, which impacts the reconstruction of normal and abnormal motions differentially, thereby enhancing their separability. Considering that motion details originate from high-frequency information, we propose a masking method based on 2D discrete cosine transform to separate high-frequency information and low-frequency information. Guided by the high-frequency information from observed motion, the diffusion model can focus on generating low-frequency information, and thus reconstructing the motion accurately. Experimental results on five video anomaly detection datasets, including human-related and open-set benchmarks, demonstrate the effectiveness of the proposed method. Our code is available at https://github.com/Xiaofeng-Tan/FGDMAD-Code.

Related papers

• Dual Conditioned Motion Diffusion for Pose-Based Video Anomaly Detection [12.100563798908777]
  Video Anomaly Detection (VAD) is essential for computer vision research. Existing VAD methods utilize either reconstruction-based or prediction-based frameworks. We address pose-based video anomaly detection and introduce a novel framework called Dual Conditioned Motion Diffusion.
  arXiv  Detail & Related papers  (2024-12-23T01:31:39Z)
• Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking [52.393359791978035]
  Motion2VecSets is a 4D diffusion model for dynamic surface reconstruction from point cloud sequences. We parameterize 4D dynamics with latent sets instead of using global latent codes. For more temporally-coherent object tracking, we synchronously denoise deformation latent sets and exchange information across multiple frames.
  arXiv  Detail & Related papers  (2024-01-12T15:05:08Z)
• Dynamic Addition of Noise in a Diffusion Model for Anomaly Detection [2.209921757303168]
  Diffusion models have found valuable applications in anomaly detection by capturing the nominal data distribution and identifying anomalies via reconstruction. Despite their merits, they struggle to localize anomalies of varying scales, especially larger anomalies such as entire missing components. We present a novel framework that enhances the capability of diffusion models, by extending the previous introduced implicit conditioning approach Meng et al. 2022 in three significant ways.
  arXiv  Detail & Related papers  (2024-01-09T09:57:38Z)
• Reconstruct-and-Generate Diffusion Model for Detail-Preserving Image Denoising [16.43285056788183]
  We propose a novel approach called the Reconstruct-and-Generate Diffusion Model (RnG) Our method leverages a reconstructive denoising network to recover the majority of the underlying clean signal. It employs a diffusion algorithm to generate residual high-frequency details, thereby enhancing visual quality.
  arXiv  Detail & Related papers  (2023-09-19T16:01:20Z)
• DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detection [38.12212015133935]
  A novel framework, DiffusionVMR, is proposed to redefine the two tasks as a unified conditional denoising generation process. Experiments conducted on five widely-used benchmarks demonstrate the effectiveness and flexibility of the proposed DiffusionVMR.
  arXiv  Detail & Related papers  (2023-08-29T08:20:23Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion. This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Exploring Diffusion Models for Unsupervised Video Anomaly Detection [17.816344808780965]
  This paper investigates the performance of diffusion models for video anomaly detection (VAD) Experiments performed on two large-scale anomaly detection datasets demonstrate the consistent improvement of the proposed method over the state-of-the-art generative models. This is the first study using a diffusion model to present guidance for examining VAD in surveillance scenarios.
  arXiv  Detail & Related papers  (2023-04-12T13:16:07Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [89.49600182243306]
  We reformulate the reconstruction process using a diffusion model into a noise-to-norm paradigm. We propose a rapid one-step denoising paradigm, significantly faster than the traditional iterative denoising in diffusion models. The segmentation sub-network predicts pixel-level anomaly scores using the input image and its anomaly-free restoration.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• The role of noise in denoising models for anomaly detection in medical images [62.0532151156057]
  Pathological brain lesions exhibit diverse appearance in brain images. Unsupervised anomaly detection approaches have been proposed using only normal data for training. We show that optimization of the spatial resolution and magnitude of the noise improves the performance of different model training regimes.
  arXiv  Detail & Related papers  (2023-01-19T21:39:38Z)
• Diffusion Models in Vision: A Survey [73.10116197883303]
  A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens.
  arXiv  Detail & Related papers  (2022-09-10T22:00:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.