A Benchmark for Crime Surveillance Video Analysis with Large Models

• URL: http://arxiv.org/abs/2502.09325v1
• Date: Thu, 13 Feb 2025 13:38:17 GMT
• Title: A Benchmark for Crime Surveillance Video Analysis with Large Models
• Authors: Haoran Chen, Dong Yi, Moyan Cao, Chensen Huang, Guibo Zhu, Jinqiao Wang,
• Abstract summary: Anomaly analysis in surveillance videos is a crucial topic in computer vision. In recent years, multimodal large language models (MLLMs) have outperformed task-specific models in various domains. We propose a benchmark for crime surveillance video analysis with large models denoted as UCVL.
• Score: 22.683394427744616
• License:
• Abstract: Anomaly analysis in surveillance videos is a crucial topic in computer vision. In recent years, multimodal large language models (MLLMs) have outperformed task-specific models in various domains. Although MLLMs are particularly versatile, their abilities to understand anomalous concepts and details are insufficiently studied because of the outdated benchmarks of this field not providing MLLM-style QAs and efficient algorithms to assess the model's open-ended text responses. To fill this gap, we propose a benchmark for crime surveillance video analysis with large models denoted as UCVL, including 1,829 videos and reorganized annotations from the UCF-Crime and UCF-Crime Annotation datasets. We design six types of questions and generate diverse QA pairs. Then we develop detailed instructions and use OpenAI's GPT-4o for accurate assessment. We benchmark eight prevailing MLLMs ranging from 0.5B to 40B parameters, and the results demonstrate the reliability of this bench. Moreover, we finetune LLaVA-OneVision on UCVL's training set. The improvement validates our data's high quality for video anomaly analysis.

Related papers

• Apollo: An Exploration of Video Understanding in Large Multimodal Models [65.06400672040836]
  We present a study that helps uncover what effectively drives video understanding in Large Multimodal Models. Our models can perceive hour-long videos efficiently, with Apollo-3B outperforming most existing $7$B models with a 55.1 on LongVideoBench. Apollo-7B is state-of-the-art compared to 7B LMMs with a 70.9 on MLVU, and 63.3 on Video-MME.
  arXiv  Detail & Related papers  (2024-12-13T18:53:24Z)
• MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding [67.56182262082729]
  We introduce MMBench-Video, a quantitative benchmark to rigorously evaluate large vision-language models (LVLMs) in video understanding. MMBench-Video incorporates lengthy videos from YouTube and employs free-form questions, mirroring practical use cases. The benchmark is meticulously crafted to probe the models' temporal reasoning skills, with all questions human-annotated according to a carefully constructed ability taxonomy.
  arXiv  Detail & Related papers  (2024-06-20T17:26:01Z)
• VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs [64.60035916955837]
  VANE-Bench is a benchmark designed to assess the proficiency of Video-LMMs in detecting anomalies and inconsistencies in videos. Our dataset comprises an array of videos synthetically generated using existing state-of-the-art text-to-video generation models. We evaluate nine existing Video-LMMs, both open and closed sources, on this benchmarking task and find that most of the models encounter difficulties in effectively identifying the subtle anomalies.
  arXiv  Detail & Related papers  (2024-06-14T17:59:01Z)
• Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis [118.08008540513596]
  Video-MME is the first-ever full-spectrum, Multi-Modal Evaluation benchmark of MLLMs in Video analysis. We extensively evaluate various state-of-the-art MLLMs, including GPT-4 series and Gemini 1.5 Pro, as well as open-source image models. Our experiments reveal that Gemini 1.5 Pro is the best-performing commercial model, significantly outperforming the open-source models.
  arXiv  Detail & Related papers  (2024-05-31T17:59:47Z)
• Harnessing Large Language Models for Training-free Video Anomaly Detection [34.76811491190446]
  Video anomaly detection (VAD) aims to temporally locate abnormal events in a video. Training-based methods are prone to be domain-specific, thus being costly for practical deployment. We propose LAnguage-based VAD (LAVAD), a method tackling VAD in a novel, training-free paradigm.
  arXiv  Detail & Related papers  (2024-04-01T09:34:55Z)
• Video Understanding with Large Language Models: A Survey [97.29126722004949]
  Given the remarkable capabilities of large language models (LLMs) in language and multimodal tasks, this survey provides a detailed overview of recent advancements in video understanding. The emergent capabilities Vid-LLMs are surprisingly advanced, particularly their ability for open-ended multi-granularity reasoning. This survey presents a comprehensive study of the tasks, datasets, benchmarks, and evaluation methodologies for Vid-LLMs.
  arXiv  Detail & Related papers  (2023-12-29T01:56:17Z)
• LLM4VG: Large Language Models Evaluation for Video Grounding [39.40610479454726]
  This paper systematically evaluates the performance of different LLMs on video grounding tasks. We propose prompt methods to integrate the instruction of VG and description from different kinds of generators. Our experimental evaluations lead to two conclusions: (i) the existing VidLLMs are still far away from achieving satisfactory video grounding performance, and more time-related video tasks should be included to further fine-tune these models.
  arXiv  Detail & Related papers  (2023-12-21T08:15:02Z)
• VindLU: A Recipe for Effective Video-and-Language Pretraining [83.49216853881595]
  This paper conducts an empirical study demystifying the most important factors in the VidL model design. Using these empirical insights, we then develop a step-by-step recipe, dubbed VindLU, for effective VidL pretraining. Our model trained using our recipe achieves comparable or better than state-of-the-art results on several VidL tasks.
  arXiv  Detail & Related papers  (2022-12-09T18:54:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.