Advance Fake Video Detection via Vision Transformers

• URL: http://arxiv.org/abs/2504.20669v1
• Date: Tue, 29 Apr 2025 11:51:07 GMT
• Title: Advance Fake Video Detection via Vision Transformers
• Authors: Joy Battocchio, Stefano Dell'Anna, Andrea Montibeller, Giulia Boato,
• Abstract summary: Vision Transformer (ViT)-based fake image detection and extend this idea to video.<n>We propose an original %innovative framework that effectively integrates ViT embeddings over time to enhance detection performance.<n>Our method shows promising accuracy, generalization, and few-shot learning capabilities across a new, large and diverse dataset of videos.
• Score: 0.9035379689195373
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in AI-based multimedia generation have enabled the creation of hyper-realistic images and videos, raising concerns about their potential use in spreading misinformation. The widespread accessibility of generative techniques, which allow for the production of fake multimedia from prompts or existing media, along with their continuous refinement, underscores the urgent need for highly accurate and generalizable AI-generated media detection methods, underlined also by new regulations like the European Digital AI Act. In this paper, we draw inspiration from Vision Transformer (ViT)-based fake image detection and extend this idea to video. We propose an {original} %innovative framework that effectively integrates ViT embeddings over time to enhance detection performance. Our method shows promising accuracy, generalization, and few-shot learning capabilities across a new, large and diverse dataset of videos generated using five open source generative techniques from the state-of-the-art, as well as a separate dataset containing videos produced by proprietary generative methods.

Related papers

• LAVID: An Agentic LVLM Framework for Diffusion-Generated Video Detection [14.687867348598035]
  Large Vision Language Model (LVLM) has become an emerging tool for AI-generated content detection. We propose LAVID, a novel LVLMs-based ai-generated video detection with explicit knowledge enhancement. Our proposed pipeline automatically selects a set of explicit knowledge tools for detection, and then adaptively adjusts the structure prompt by self-rewriting.
  arXiv  Detail & Related papers  (2025-02-20T19:34:58Z)
• Enhance-A-Video: Better Generated Video for Free [57.620595159855064]
  We introduce a training-free approach to enhance the coherence and quality of DiT-based generated videos.<n>Our approach can be easily applied to most DiT-based video generation frameworks without any retraining or fine-tuning.
  arXiv  Detail & Related papers  (2025-02-11T12:22:35Z)
• DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation [54.30327187663316]
  DiTCtrl is a training-free multi-prompt video generation method under MM-DiT architectures for the first time. We analyze MM-DiT's attention mechanism, finding that the 3D full attention behaves similarly to that of the cross/self-attention blocks in the UNet-like diffusion models. Based on our careful design, the video generated by DiTCtrl achieves smooth transitions and consistent object motion given multiple sequential prompts.
  arXiv  Detail & Related papers  (2024-12-24T18:51:19Z)
• The Tug-of-War Between Deepfake Generation and Detection [4.62070292702111]
  Multimodal generative models are rapidly evolving, leading to a surge in the generation of realistic video and audio. Deepfake videos, which can convincingly impersonate individuals, have particularly garnered attention due to their potential misuse. This survey paper examines the dual landscape of deepfake video generation and detection, emphasizing the need for effective countermeasures.
  arXiv  Detail & Related papers  (2024-07-08T17:49:41Z)
• WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs [53.21307319844615]
  We present an innovative video generation AI agent that harnesses the power of Sora-inspired multimodal learning to build skilled world models framework. The framework includes two parts: prompt enhancer and full video translation.
  arXiv  Detail & Related papers  (2024-03-10T16:09:02Z)
• A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming [26.082980156232086]
  Top-trending AI technologies, i.e., generative artificial intelligence (Generative AI) and large language models (LLMs), are reshaping the field of video technology. The paper highlights the innovative use of these technologies in producing highly realistic videos. In the realm of video streaming, the paper discusses how LLMs contribute to more efficient and user-centric streaming experiences.
  arXiv  Detail & Related papers  (2024-01-30T14:37:10Z)
• AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection [53.448283629898214]
  The recent proliferation of hyper-realistic deepfake videos has drawn attention to the threat of audio and visual forgeries. Most previous work on detecting AI-generated fake videos only utilize visual modality or audio modality. We propose an Audio-Visual Transformer-based Ensemble Network (AVTENet) framework that considers both acoustic manipulation and visual manipulation.
  arXiv  Detail & Related papers  (2023-10-19T19:01:26Z)
• Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learning [50.60891619269651]
  Control-A-Video is a controllable T2V diffusion model that can generate videos conditioned on text prompts and reference control maps like edge and depth maps. We propose novel strategies to incorporate content prior and motion prior into the diffusion-based generation process. Our framework generates higher-quality, more consistent videos compared to existing state-of-the-art methods in controllable text-to-video generation.
  arXiv  Detail & Related papers  (2023-05-23T09:03:19Z)
• Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models [149.1331903899298]
  We propose a novel framework called BIKE, which utilizes the cross-modal bridge to explore bidirectional knowledge. We present a Temporal Concept Spotting mechanism that uses the Text-to-Video expertise to capture temporal saliency in a parameter-free manner. Our best model achieves a state-of-the-art accuracy of 88.6% on the challenging Kinetics-400 using the released CLIP model.
  arXiv  Detail & Related papers  (2022-12-31T11:36:53Z)
• Hybrid Transformer Network for Deepfake Detection [2.644723682054489]
  We propose a novel hybrid transformer network utilizing early feature fusion strategy for deepfake video detection. Our model achieves comparable results to other more advanced state-of-the-art approaches when evaluated on FaceForensics++ and DFDC benchmarks. We also propose novel face cut-out augmentations, as well as random cut-out augmentations.
  arXiv  Detail & Related papers  (2022-08-11T13:30:42Z)
• TiVGAN: Text to Image to Video Generation with Step-by-Step Evolutionary Generator [34.7504057664375]
  We propose a novel training framework, Text-to-Image-to-Video Generative Adversarial Network (TiVGAN), which evolves frame-by-frame and finally produces a full-length video. Step-by-step learning process helps stabilize the training and enables the creation of high-resolution video based on conditional text descriptions.
  arXiv  Detail & Related papers  (2020-09-04T06:33:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.