Next-Frame Feature Prediction for Multimodal Deepfake Detection and Temporal Localization

• URL: http://arxiv.org/abs/2511.10212v1
• Date: Fri, 14 Nov 2025 01:39:34 GMT
• Title: Next-Frame Feature Prediction for Multimodal Deepfake Detection and Temporal Localization
• Abstract summary: We propose a single-stage training framework that enhances generalization.<n>We introduce a window-level attention mechanism to capture discrepancies between predicted and actual frames.<n>Our model, evaluated on multiple benchmark datasets, demonstrates strong generalization and precise temporal localization.
• Score: 37.361231344742045
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent multimodal deepfake detection methods designed for generalization conjecture that single-stage supervised training struggles to generalize across unseen manipulations and datasets. However, such approaches that target generalization require pretraining over real samples. Additionally, these methods primarily focus on detecting audio-visual inconsistencies and may overlook intra-modal artifacts causing them to fail against manipulations that preserve audio-visual alignment. To address these limitations, we propose a single-stage training framework that enhances generalization by incorporating next-frame prediction for both uni-modal and cross-modal features. Additionally, we introduce a window-level attention mechanism to capture discrepancies between predicted and actual frames, enabling the model to detect local artifacts around every frame, which is crucial for accurately classifying fully manipulated videos and effectively localizing deepfake segments in partially spoofed samples. Our model, evaluated on multiple benchmark datasets, demonstrates strong generalization and precise temporal localization.

Related papers

• EVAS: Efficient Multimodal Temporal Forgery Localization via Audio-Visual Synergy and Steered Boundary Calibration [14.088182834458843]
  We propose EVAS, an end-to-end framework for temporal forgery localization.<n>We introduce a Boundary-Aware Refinement strategy to achieve steered boundary calibration.<n> EVAS achieves state-of-the-art average localization accuracy and average recall across three benchmark datasets.
  arXiv  Detail & Related papers  (2026-07-05T19:33:21Z)
• Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding [56.38477453373368]
  LazyMCoT is a dynamic and training-free framework that adaptively allocates visual grounding efforts based on sample difficulty.<n>LazyMCoT rivals training-based approaches by simultaneously improving reasoning accuracy and reducing average inference latency.
  arXiv  Detail & Related papers  (2026-06-15T03:17:44Z)
• FakeRadar: Probing Forgery Outliers to Detect Unknown Deepfake Videos [56.782373316571444]
  We propose FakeRadar, a novel deepfake video detection framework.<n>FakeRadar addresses the challenges of cross-domain generalization in real-world scenarios.
  arXiv  Detail & Related papers  (2025-12-16T17:11:45Z)
• Multi-modal Deepfake Detection and Localization with FPN-Transformer [21.022230340898556]
  We introduce a multi-modal deepfake detection and localization framework based on a Feature Pyramid-Transformer (FPN-Transformer)<n>A multi-scale feature pyramid is constructed through R-TLM blocks with localized attention mechanisms, enabling joint analysis of cross-context temporal dependencies.<n>We evaluate our approach on the test set of the IJCAI'25 DDL-AV benchmark, showing a good performance with a final score of 0.7535.
  arXiv  Detail & Related papers  (2025-11-11T09:33:39Z)
• Detecting Audio-Visual Deepfakes with Fine-Grained Inconsistencies [11.671275975119089]
  We propose the introduction of fine-grained mechanisms for detecting subtle artifacts in both spatial and temporal domains. First, we introduce a local audio-visual model capable of capturing small spatial regions that are prone to inconsistencies with audio. Second, we introduce a temporally-local pseudo-fake augmentation to include samples incorporating subtle temporal inconsistencies in our training set.
  arXiv  Detail & Related papers  (2024-08-13T09:19:59Z)
• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• GM-DF: Generalized Multi-Scenario Deepfake Detection [49.072106087564144]
  Existing face forgery detection usually follows the paradigm of training models in a single domain. In this paper, we elaborately investigate the generalization capacity of deepfake detection models when jointly trained on multiple face forgery detection datasets.
  arXiv  Detail & Related papers  (2024-06-28T17:42:08Z)
• Benchmarking Cross-Domain Audio-Visual Deception Detection [45.342156006617394]
  We present the first cross-domain audio-visual deception detection benchmark.<n>We compare single-to-single and multi-to-single domain generalization performance.<n>We propose an algorithm to enhance the generalization performance.
  arXiv  Detail & Related papers  (2024-05-11T12:06:31Z)
• Training-Free Deepfake Voice Recognition by Leveraging Large-Scale Pre-Trained Models [52.04189118767758]
  Generalization is a main issue for current audio deepfake detectors. In this paper we study the potential of large-scale pre-trained models for audio deepfake detection.
  arXiv  Detail & Related papers  (2024-05-03T15:27:11Z)
• Point-Level Temporal Action Localization: Bridging Fully-supervised Proposals to Weakly-supervised Losses [84.2964408497058]
  Point-level temporal action localization (PTAL) aims to localize actions in untrimmed videos with only one timestamp annotation for each action instance. Existing methods adopt the frame-level prediction paradigm to learn from the sparse single-frame labels. This paper attempts to explore the proposal-based prediction paradigm for point-level annotations.
  arXiv  Detail & Related papers  (2020-12-15T12:11:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.