Dataset Biases and Shortcut Learning in Motion-Based AI-Generated Video Detection

• URL: http://arxiv.org/abs/2607.00948v1
• Date: Wed, 01 Jul 2026 13:47:42 GMT
• Title: Dataset Biases and Shortcut Learning in Motion-Based AI-Generated Video Detection
• Abstract summary: We evaluate the robustness and applicability of four state-of-the-art motion-based AI-generated video detectors.<n>We identify significant preprocessing and sampling biases in these methods and demonstrate that they account for a substantial portion of their reported performance.<n>We show that for all detectors, performance collapses to near-random levels when evaluated on a dataset that does not contain this motion bias.
• Score: 0.8092772265574579
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The visual quality of AI-generated videos has improved drastically in recent years, making it increasingly difficult for humans to distinguish between real and synthetic media. In this work, we evaluate the robustness and applicability of four state-of-the-art motion-based AI-generated video detectors. We identify significant preprocessing and sampling biases in these methods and demonstrate that they account for a substantial portion of their reported performance. Furthermore, we find that these detectors are highly sensitive to motion patterns specific to their evaluation datasets, where AI-generated videos generally exhibit less inter-frame movement than real videos. We show that for all detectors, performance collapses to near-random levels when evaluated on a dataset that does not contain this motion bias. Additionally, through dataset rebalancing and the application of simple spatial augmentations, we observe severe performance degradation across all evaluated models. In contrast, we find that an existing frequency-based detector maintains strong performance across all evaluated datasets, suggesting that frequency-based approaches may offer a more generalizable path forward for AI-generated video detection. We hope that our work raises awareness towards these vulnerabilities and encourages the development of more representative, unbiased datasets and more robust evaluation protocols.

Related papers

• SphereVideo: Prototype-anchored Hyperspherical Boundary for Continual AI-generated Video Detection [61.05420454815012]
  AI-generated video (AIGV) detection aims to distinguish real videos from AI-generated ones.<n>In practice, detectors trained on existing data often fail to generalize to newly emerging generative models.<n>We propose SphereVideo, a novel framework for AIGV detection built on two key observations.
  arXiv  Detail & Related papers  (2026-08-02T15:54:54Z)
• Preserving Forgery Artifacts: AI-Generated Video Detection at Native Scale [36.51885181895111]
  We introduce a comprehensive dataset and a novel detection framework.<n>First, we curate a large-scale dataset of over 140K videos from 15 state-of-the-art open-source and commercial generators.<n>We propose a novel detection framework built on the Qwen-VL Vision Transformer, which operates at variable spatial resolutions and temporal durations.
  arXiv  Detail & Related papers  (2026-04-06T12:36:08Z)
• GV-VAD : Exploring Video Generation for Weakly-Supervised Video Anomaly Detection [6.09434007746295]
  Video anomaly detection (VAD) plays a critical role in public safety applications such as intelligent surveillance.<n>We propose a generative video-enhanced weakly-supervised VAD framework to produce semantically controllable and physically plausible synthetic videos.<n>The proposed framework outperforms state-of-the-art methods on UCF-Crime datasets.
  arXiv  Detail & Related papers  (2025-08-01T04:42:40Z)
• Leveraging Pre-Trained Visual Models for AI-Generated Video Detection [54.88903878778194]
  The field of video generation has advanced beyond DeepFakes, creating an urgent need for methods capable of detecting AI-generated videos with generic content.<n>We propose a novel approach that leverages pre-trained visual models to distinguish between real and generated videos.<n>Our method achieves high detection accuracy, above 90% on average, underscoring its effectiveness.
  arXiv  Detail & Related papers  (2025-07-17T15:36:39Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• AI-Generated Video Detection via Perceptual Straightening [10.189379841180285]
  We propose ReStraV, a novel approach to distinguish natural from AI-generated videos.<n>Inspired by the "perceptual straightening" hypothesis, we quantify the temporal curvature and stepwise distance in the model's representation domain.<n>Our analysis shows that AI-generated videos exhibit significantly different curvature and distance patterns compared to real videos.
  arXiv  Detail & Related papers  (2025-07-01T09:04:21Z)
• BrokenVideos: A Benchmark Dataset for Fine-Grained Artifact Localization in AI-Generated Videos [63.03271511550633]
  BrokenVideos is a benchmark dataset of 3,254 AI-generated videos with meticulously annotated, pixel-level masks highlighting regions of visual corruption.<n>Our experiments show that training state of the art artifact detection models and multi modal large language models (MLLMs) on BrokenVideos significantly improves their ability to localize corrupted regions.
  arXiv  Detail & Related papers  (2025-06-25T03:30:04Z)
• Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation [31.737159092430108]
  We study different generative architectures, searching and identifying discriminative features that are unbiased, robust to impairments, and shared across models.<n>We introduce a novel data augmentation strategy based on the wavelet decomposition and replace specific frequency-related bands to drive the model to exploit more relevant forensic cues.<n>Our method achieves a significant accuracy improvement over state-of-the-art detectors and obtains excellent results even on very recent generative models.
  arXiv  Detail & Related papers  (2025-06-20T07:36:59Z)
• SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learning [78.44705665291741]
  We present a comprehensive evaluation of modern video self-supervised models.<n>We focus on generalization across four key downstream factors: domain shift, sample efficiency, action granularity, and task diversity.<n>Our analysis shows that, despite architectural advances, transformer-based models remain sensitive to downstream conditions.
  arXiv  Detail & Related papers  (2025-04-08T06:00:28Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models.<n>In this paper, we investigate how detection performance varies across model backbones, types, and datasets.<n>We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Exposing AI-generated Videos: A Benchmark Dataset and a Local-and-Global Temporal Defect Based Detection Method [31.763312726582217]
  generative model has made significant advancements in the creation of realistic videos, which causes security issues. In this paper, we first construct a video dataset using advanced diffusion-based video generation algorithms with various semantic contents. By analyzing local and global temporal defects of current AI-generated videos, a novel detection framework is constructed to expose fake videos.
  arXiv  Detail & Related papers  (2024-05-07T09:00:09Z)
• Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks [76.35271072704384]
  Deep learning models perform poorly when applied to videos with rare scenes or objects. We tackle this problem from two different angles: algorithm and dataset. We show that the debiased representation can generalize better when transferred to other datasets and tasks.
  arXiv  Detail & Related papers  (2022-09-20T00:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.