Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit

• URL: http://arxiv.org/abs/2606.31004v1
• Date: Tue, 30 Jun 2026 00:29:53 GMT
• Title: Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit
• Abstract summary: AI-generated video detection GenVidBench and AIGVDBench are the de facto leaderboards.<n>Most evaluation protocols leave uncontrolled confounds that can inflate reported generalizations.<n>A 20-paper survey finds none applying all six standard controls that would catch this.<n>We combine them into an audited protocol and apply it to six representative feature sources.
• Score: 8.665845754660458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI-generated video detection benchmarks such as GenVidBench and AIGVDBench are the de facto leaderboards, yet most evaluation protocols leave uncontrolled confounds that can inflate reported generalization. As an existence proof, a three-feature clip-length classifier reaches a leave-one-generator-out (LOGO) AUC of 0.998 on GenVidBench under unaudited evaluation, while measuring nothing about motion. A 20-paper survey finds none applying all six standard controls that would catch this, so we combine them into an audited protocol and apply it to six representative feature sources (three published detectors and three repurposed signal sources), re-running it cross-dataset on AIGVDBench. The audit both debunks and certifies: the trivial classifier collapses to near chance (0.529), a CLIP baseline is caught carrying dataset identity, and the 2025 forensic detector WaveRep clears the floor at out-of-distribution LOGO AUC 0.996 with chance-level real-vs-real coherence. At a deployable FPR of 0.1%, multiple high-AUC methods fall to single-digit recall and the leaderboard order changes, so we recommend an audited tuple (AUC, above-floor margin, operating-point recall, and calibration) over a single number. As a white-box positive control, we add TemporalSpec (codec motion vectors); via cross-substrate feature fusion (XSFF), a second substrate adds genuine complementarity that survives the audit. We release VidAudit, to our knowledge the largest unified and audited detector collection for this task, providing 14 detectors behind one plugin API, a leaderboard, and Croissant metadata, available at https://github.com/KurbanIntelligenceLab/vidaudit. Together, the protocol and toolkit move evaluation from leaderboard rank toward whether a result measures what it claims.

Related papers

• ArguLens: An Open-Source System for Automated Essay Scoring and Label-Aware Feedback Generation [9.677955445684686]
  ArguLens is an opensource system that decomposes automated essay scoring (AES) into three decoupled components.<n>A label-aware feedback generator serves through vLLM with a Qwen2.5-14BInstruct backbone.<n>A Gradio web UI exposes pluggable inference backends and supports single-essay and batch scoring.
  arXiv  Detail & Related papers  (2026-08-18T04:23:50Z)
• Graph Is the Verifier: Agentic Reinforcement Learning for Interprocedural Vulnerability Detection [13.409515289348532]
  VulAgentRL is a framework for interprocedural vulnerability detection built on a Code Property Graph (CPG)<n>We show that VulAgentRL outperforms state-of-the-art baselines, including frontier models, on the strict pair-wise-correct metric while issuing fewer tool calls.
  arXiv  Detail & Related papers  (2026-07-29T09:16:54Z)
• Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion [46.262619407930266]
  We train a multi-label classifier on a curated gold dataset of 1,207 CVEs from expert MITRE Center for Threat-Informed Defense mappings.<n>The resulting model approximately doubles recall@5 compared with a zero-shot embedding-similarity baseline.<n>We then investigate whether LLM-assisted labeling can extend the gold dataset.
  arXiv  Detail & Related papers  (2026-07-28T10:59:04Z)
• DynaKRAG: A Unified Framework for Learnable Evidence Control in Multi-Hop Retrieval-Augmented Generation [12.710406983346013]
  DynaKRAG formulates multi-hop evidence acquisition as state-conditioned control over atomic evidence operations.<n>With Qwen2.5-7B-Instruct, DynaKRAG achieves F1 scores of 0.5998 on HotpotQA, 0.5340 on 2Wiki, and 0.3061 on MuSiQue.
  arXiv  Detail & Related papers  (2026-07-07T17:09:36Z)
• Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing [22.30404793367838]
  We introduce STATEWITNESS, an activation explainer for deception auditing.<n>A separate decoder reads a target model's hidden states, then answers natural-language queries or emits structured reports about them.<n>We evaluate STATEWITNESS on two target reasoning LLMs across seven deception datasets.
  arXiv  Detail & Related papers  (2026-06-16T03:41:29Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Pretrained, Frozen, Still Leaking: Auditing Cross-Encoder Attribute Transfer in EEG Foundation Models [0.0]
  We show that each single-endpoint audit clears releases that still leak spectral attributes.<n>The contribution is an audit framework that turns scattered single-endpoint defenses into a joint release decision.
  arXiv  Detail & Related papers  (2026-06-08T08:24:03Z)
• PInVerify: An Offline Embodied Benchmark for Active Instance Verification [8.403971471573607]
  Embodied agents have made strong progress in navigating to target objects, but reaching the goal vicinity does not guarantee that the agent has found the correct instance.<n>We address this gap with Active Instance Verification (AIV), a task in which an agent actively selects viewpoints around a candidate object to decide whether it matches a fine-grained natural-language description.<n>We introduce PInVerify, an offline embodied benchmark for AIV: 3,000 evaluation episodes across 18 object categories, delivered as multi-view captures.
  arXiv  Detail & Related papers  (2026-05-28T22:42:38Z)
• SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation [6.604874054866016]
  We present SURE-RAG, a transparent aggregation protocol built on the observation that evidence sufficiency is a set-level property.<n>A shared pair-level claim-evidence verifier produces local relation distributions, which SURE-RAG aggregates into interpretable answer-level signals.<n>We evaluate on HotpotQA-RAG v3, a controlled multi-hop benchmark, under an artifact-aware protocol.
  arXiv  Detail & Related papers  (2026-05-05T09:05:40Z)
• Governing Dynamic Capabilities: Cryptographic Binding and Reproducibility Verification for AI Agent Tool Use [0.0]
  No existing security layer verifies what an AI agent can do, whether it executed what it claims, or what happened in a multi-agent interaction.<n>Existing frameworks conflate the two, enabling silent capability escalation and leaving interactions without verifiable provenance.<n>We derive three Agent Governance Requirements: capability integrity (G1), behavioral verifiability (G2), and interaction auditability (G3)<n>We validate with two crypto-agnostic instantiations -- basic (Ed25519, SHA-256; 97 us verify) and enhanced (BBS+ selective disclosure, Groth16 DV-SNARK; 13.8 ms)
  arXiv  Detail & Related papers  (2026-03-15T11:46:57Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• Box-Level Active Detection [47.41635810670186]
  We introduce a box-level active detection framework that controls a box-based budget per cycle. We propose Complementary Pseudo Active Strategy (ComPAS) to exploit both human annotations and the model intelligence. ComPAS consistently outperforms 10 competitors under 4 settings in a unified setting.
  arXiv  Detail & Related papers  (2023-03-23T08:06:10Z)
• Disentangle Your Dense Object Detector [82.22771433419727]
  Deep learning-based dense object detectors have achieved great success in the past few years and have been applied to numerous multimedia applications such as video understanding. However, the current training pipeline for dense detectors is compromised to lots of conjunctions that may not hold. We propose Disentangled Dense Object Detector (DDOD), in which simple and effective disentanglement mechanisms are designed and integrated into the current state-of-the-art detectors.
  arXiv  Detail & Related papers  (2021-07-07T00:52:16Z)
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement [53.69674636044927]
  We present EHSOD, an end-to-end hybrid-supervised object detection system. It can be trained in one shot on both fully and weakly-annotated data. It achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data.
  arXiv  Detail & Related papers  (2020-02-18T08:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.