Real-Time Video Anomaly Detection Using YOLO Pose Estimation and CLIP-Based Semantic Scoring

• URL: http://arxiv.org/abs/2608.31074v1
• Date: Mon, 31 Aug 2026 16:47:29 GMT
• Title: Real-Time Video Anomaly Detection Using YOLO Pose Estimation and CLIP-Based Semantic Scoring
• Abstract summary: We propose a lightweight two-stage framework for real-time video anomaly detection.<n>The first stage employs YOLO v11n-pose to detect persons and extract seventeen skeletal keypoints in a single forward pass.<n>The second stage encodes each cropped person region through CLIP ViT-B/32 and computes cosine similarity.
• Score: 0.3784545047113015
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a lightweight two-stage framework for real-time video anomaly detection. The first stage employs YOLO v11n-pose to detect persons and extract seventeen skeletal keypoints in a single forward pass. The second stage encodes each cropped person region through CLIP ViT-B/32 and computes cosine similarity against predefined textual descriptions of anomalous behaviors. This architecture eliminates the need for optical flow, standalone pose estimators, and density-based scoring modules. Experiments on CUHK Avenue, ShanghaiTech Campus, and a custom indoor dataset collected at Chulalongkorn University demonstrate an end-to-end throughput of approximately 51 FPS on an NVIDIA Titan XP GPU, a 3.36x speedup over the multi-feature baseline, while maintaining frame-level AUROC values of 89.26%, 70.26%, and 84.13%, respectively.

Related papers

• LEADER: Lightweight End-to-End Attention-Gated Dual Autoencoder for Robust Minutiae Extraction [0.05978532290288763]
  This paper introduces LEADER (Lightweight End-to-end Attention-gated Dual autoencodER), a neural network that maps raw fingerprint images to minutiae descriptors.<n>It employs a novel "Castle-Moat-Rampart" ground-truth encoding and a dual-autoencoder structure, interconnected through an attention-gating mechanism.<n>It attains a 34% higher F1-score on the NIST SD27 dataset compared to specialized latent minutiae extractors.
  arXiv  Detail & Related papers  (2026-02-17T11:02:28Z)
• T*: Re-thinking Temporal Search for Long-Form Video Understanding [66.72243342954823]
  Current temporal search methods only achieve 2.1% temporal F1 score on the Longvideobench subset.<n>Inspired by visual search in images, we propose a lightweight temporal search framework, T* that reframes costly temporal search as spatial search.<n>Extensive experiments show that integrating T* with existing methods significantly improves SOTA long-form video understanding.
  arXiv  Detail & Related papers  (2025-04-03T04:03:10Z)
• 8-Calves Image dataset [0.8233028449337972]
  We introduce the 8-Calves dataset, a challenging benchmark for multi-animal detection, tracking, and identification.<n>It features a one-hour video of eight Holstein Friesian calves in a barn, with frequent occlusions, motion blur, and diverse poses.<n>A semi-grained pipeline using a fine-tuned YOLOv8 detector and ByteTrack, followed by manual correction, provides over 537,000 bounding boxes with temporal identity labels.
  arXiv  Detail & Related papers  (2025-03-17T23:47:52Z)
• Practical Video Object Detection via Feature Selection and Aggregation [18.15061460125668]
  Video object detection (VOD) needs to concern the high across-frame variation in object appearance, and the diverse deterioration in some frames. Most of contemporary aggregation methods are tailored for two-stage detectors, suffering from high computational costs. This study invents a very simple yet potent strategy of feature selection and aggregation, gaining significant accuracy at marginal computational expense.
  arXiv  Detail & Related papers  (2024-07-29T02:12:11Z)
• SimulFlow: Simultaneously Extracting Feature and Identifying Target for Unsupervised Video Object Segmentation [28.19471998380114]
  Unsupervised video object segmentation (UVOS) aims at detecting the primary objects in a given video sequence without any human interposing. Most existing methods rely on two-stream architectures that separately encode the appearance and motion information before fusing them to identify the target and generate object masks. We propose a novel UVOS model called SimulFlow that simultaneously performs feature extraction and target identification.
  arXiv  Detail & Related papers  (2023-11-30T06:44:44Z)
• Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors [117.61449210940955]
  We propose an efficient abnormal event detection model based on a lightweight masked auto-encoder (AE) applied at the video frame level. We introduce an approach to weight tokens based on motion gradients, thus shifting the focus from the static background scene to the foreground objects. We generate synthetic abnormal events to augment the training videos, and task the masked AE model to jointly reconstruct the original frames.
  arXiv  Detail & Related papers  (2023-06-21T06:18:05Z)
• TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement [64.11385310305612]
  We present a novel model for Tracking Any Point (TAP) that effectively tracks any queried point on any physical surface throughout a video sequence. Our approach employs two stages: (1) a matching stage, which independently locates a suitable candidate point match for the query point on every other frame, and (2) a refinement stage, which updates both the trajectory and query features based on local correlations. The resulting model surpasses all baseline methods by a significant margin on the TAP-Vid benchmark, as demonstrated by an approximate 20% absolute average Jaccard (AJ) improvement on DAVIS.
  arXiv  Detail & Related papers  (2023-06-14T17:07:51Z)
• Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection [20.161887223481994]
  We propose a long-sequence modeling framework, named StreamPETR, for multi-view 3D object detection. StreamPETR achieves significant performance improvements only with negligible cost, compared to the single-frame baseline. The lightweight version realizes 45.0% mAP and 31.7 FPS, outperforming the state-of-the-art method (SOLOFusion) by 2.3% mAP and 1.8x faster FPS.
  arXiv  Detail & Related papers  (2023-03-21T15:19:20Z)
• Negative Frames Matter in Egocentric Visual Query 2D Localization [119.23191388798921]
  Recently released Ego4D dataset and benchmark significantly scales and diversifies first-person visual perception data. Visual Queries 2D localization task aims to retrieve objects appeared in the past from the recording in the first-person view. Our study is based on the three-stage baseline introduced in the Episodic Memory benchmark.
  arXiv  Detail & Related papers  (2022-08-03T09:54:51Z)
• Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation [61.98690211671168]
  We propose a Multi-level Attention-Decoder Network (MAED) to model multi-level attentions in a unified framework. With the training set of 3DPW, MAED outperforms previous state-of-the-art methods by 6.2, 7.2, and 2.4 mm of PA-MPJPE.
  arXiv  Detail & Related papers  (2021-09-06T09:06:17Z)
• Real-time Face Mask Detection in Video Data [0.5371337604556311]
  We present a robust deep learning pipeline that is capable of identifying correct and incorrect mask-wearing from real-time video streams. We devised two separate approaches and evaluated their performance and run-time efficiency.
  arXiv  Detail & Related papers  (2021-05-05T01:03:34Z)
• Spatiotemporal Contrastive Video Representation Learning [87.56145031149869]
  We present a self-supervised Contrastive Video Representation Learning (CVRL) method to learn visual representations from unlabeled videos. Our representations are learned using a contrasttemporalive loss, where two augmented clips from the same short video are pulled together in the embedding space. We study what makes for good data augmentations for video self-supervised learning and find that both spatial and temporal information are crucial.
  arXiv  Detail & Related papers  (2020-08-09T19:58:45Z)
• Tracking Objects as Points [83.9217787335878]
  We present a simultaneous detection and tracking algorithm that is simpler, faster, and more accurate than the state of the art. Our tracker, CenterTrack, applies a detection model to a pair of images and detections from the prior frame. CenterTrack is simple, online (no peeking into the future), and real-time.
  arXiv  Detail & Related papers  (2020-04-02T17:58:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.