LoCC: Detection and Localization of Lip-Syncing Deepfakes via Counterfactual Frame Consistency

• URL: http://arxiv.org/abs/2606.22772v1
• Date: Mon, 22 Jun 2026 02:19:39 GMT
• Title: LoCC: Detection and Localization of Lip-Syncing Deepfakes via Counterfactual Frame Consistency
• Abstract summary: Lip-syncing deepfakes are among the most challenging forms of manipulated media because their artifacts are localized almost exclusively to the mouth region.<n>We propose LoCC, a novel detection framework that performs fine-grained detection and localization of lip-syncing deepfakes at both segment and frame levels.
• Score: 28.817029705247155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lip-syncing deepfakes are among the most challenging forms of manipulated media because their artifacts are localized almost exclusively to the mouth region and evolve dynamically over time. Detecting such deepfakes requires precise temporal and spatial modeling of lip motion. In this paper, we propose LoCC, a novel detection framework that performs fine-grained detection and localization of lip-syncing deepfakes at both segment and frame levels. Unlike prior approaches that analyze videos holistically, our method evaluates whether each frame aligns with a counterfactual estimate generated from its temporal neighbors. Real videos exhibit strong and stable consistency, whereas lip-sync deepfakes introduce localized inconsistencies. Following a teacher-student learning paradigm, our model effectively captures these frame-level discrepancies and achieves superior performance over state-of-the-art methods on multiple benchmark lip-syncing deepfake datasets, including LAV-DF, AVDF1M, FakeAVCeleb, and KODF, and generalizes well across compression levels and datasets.

Related papers

• Video Depth Propagation [54.523028170425256]
  Existing methods rely on simple frame-by-frame monocular models, leading to temporal inconsistencies and inaccuracies.<n>We propose VeloDepth, which effectively leverages an online video pipeline and performs deep feature propagation.<n>Our design structurally enforces temporal consistency, resulting in stable depth predictions across consecutive frames with improved efficiency.
  arXiv  Detail & Related papers  (2025-12-11T15:08:37Z)
• Multi-modal Deepfake Detection and Localization with FPN-Transformer [21.022230340898556]
  We introduce a multi-modal deepfake detection and localization framework based on a Feature Pyramid-Transformer (FPN-Transformer)<n>A multi-scale feature pyramid is constructed through R-TLM blocks with localized attention mechanisms, enabling joint analysis of cross-context temporal dependencies.<n>We evaluate our approach on the test set of the IJCAI'25 DDL-AV benchmark, showing a good performance with a final score of 0.7535.
  arXiv  Detail & Related papers  (2025-11-11T09:33:39Z)
• Frame-level Temporal Difference Learning for Partial Deepfake Speech Detection [16.923285534924116]
  We propose a Temporal Difference Attention Module (TDAM) that redefines partial deepfake detection as identifying unnatural temporal variations.<n>A dual-level hierarchical difference representation captures temporal irregularities at both fine and coarse scales, while adaptive average pooling preserves essential patterns across variable-length inputs to minimize information loss.<n>Our TDAM-AvgPool model achieves state-of-the-art performance, with an EER of 0.59% on the PartialSpoof dataset and 0.03% on the HAD dataset, which significantly outperforms the existing methods without requiring frame-level supervision.
  arXiv  Detail & Related papers  (2025-07-20T19:46:23Z)
• Context-aware TFL: A Universal Context-aware Contrastive Learning Framework for Temporal Forgery Localization [60.73623588349311]
  We propose a universal context-aware contrastive learning framework (UniCaCLF) for temporal forgery localization.<n>Our approach leverages supervised contrastive learning to discover and identify forged instants by means of anomaly detection.<n>An efficient context-aware contrastive coding is introduced to further push the limit of instant feature distinguishability between genuine and forged instants.
  arXiv  Detail & Related papers  (2025-06-10T06:40:43Z)
• Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistencies [29.81606633121959]
  Lip-syncing deepfakes are one of the most challenging deepfakes to detect.<n>We propose LIPINC-V2, a novel framework to detect lip-syncing deepfakes.<n>Our model can successfully capture both short-term and long-term variations in mouth movement.
  arXiv  Detail & Related papers  (2025-04-02T08:24:06Z)
• Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detection [41.35861722481721]
  Deepfake threats to society and cybersecurity have provoked significant public apprehension. This paper introduces an elegantly simple yet effective strategy named Thumbnail Layout (TALL) TALL transforms a video clip into a pre-defined layout to realize the preservation of spatial and temporal dependencies.
  arXiv  Detail & Related papers  (2024-03-15T12:48:44Z)
• Exposing Lip-syncing Deepfakes from Mouth Inconsistencies [29.81606633121959]
  A lip-syncing deepfake is a digitally manipulated video in which a person's lip movements are created convincingly using AI models to match altered or entirely new audio. In this paper, we describe a novel approach, LIP-syncing detection based on mouth INConsistency (LIPINC) for lip-syncing deepfake detection.
  arXiv  Detail & Related papers  (2024-01-18T16:35:37Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Delving into Sequential Patches for Deepfake Detection [64.19468088546743]
  Recent advances in face forgery techniques produce nearly untraceable deepfake videos, which could be leveraged with malicious intentions. Previous studies has identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods. We propose the Local- & Temporal-aware Transformer-based Deepfake Detection framework, which adopts a local-to-global learning protocol.
  arXiv  Detail & Related papers  (2022-07-06T16:46:30Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection [118.37239586697139]
  LipForensics is a detection approach capable of both generalising manipulations and withstanding various distortions. It consists in first pretraining a-temporal network to perform visual speech recognition (lipreading) A temporal network is subsequently finetuned on fixed mouth embeddings of real and forged data in order to detect fake videos based on mouth movements without over-fitting to low-level, manipulation-specific artefacts.
  arXiv  Detail & Related papers  (2020-12-14T15:53:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.