Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization

• URL: http://arxiv.org/abs/2601.21458v1
• Date: Thu, 29 Jan 2026 09:35:27 GMT
• Title: Mining Forgery Traces from Reconstruction Error: A Weakly Supervised Framework for Multimodal Deepfake Temporal Localization
• Abstract summary: We propose a weakly supervised forgery-based temporal localization framework that identifies forgeries via reconstruction errors.<n>Our framework uses a Masked Autoencoder (MAE) trained exclusively on authentic data to learn intrinsic its patterns.<n>We show that RT-Deepfake achieves state-of-the-art performance in weakly-supervised temporal localization.
• Score: 25.609186178759995
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern deepfakes have evolved into localized and intermittent manipulations that require fine-grained temporal localization. The prohibitive cost of frame-level annotation makes weakly supervised methods a practical necessity, which rely only on video-level labels. To this end, we propose Reconstruction-based Temporal Deepfake Localization (RT-DeepLoc), a weakly supervised temporal forgery localization framework that identifies forgeries via reconstruction errors. Our framework uses a Masked Autoencoder (MAE) trained exclusively on authentic data to learn its intrinsic spatiotemporal patterns; this allows the model to produce significant reconstruction discrepancies for forged segments, effectively providing the missing fine-grained cues for localization. To robustly leverage these indicators, we introduce a novel Asymmetric Intra-video Contrastive Loss (AICL). By focusing on the compactness of authentic features guided by these reconstruction cues, AICL establishes a stable decision boundary that enhances local discrimination while preserving generalization to unseen forgeries. Extensive experiments on large-scale datasets, including LAV-DF, demonstrate that RT-DeepLoc achieves state-of-the-art performance in weakly-supervised temporal forgery localization.

Related papers

• EVAS: Efficient Multimodal Temporal Forgery Localization via Audio-Visual Synergy and Steered Boundary Calibration [14.088182834458843]
  We propose EVAS, an end-to-end framework for temporal forgery localization.<n>We introduce a Boundary-Aware Refinement strategy to achieve steered boundary calibration.<n> EVAS achieves state-of-the-art average localization accuracy and average recall across three benchmark datasets.
  arXiv  Detail & Related papers  (2026-07-05T19:33:21Z)
• Learning Where and When: Patch-Based Spatiotemporal Localization in Weakly Supervised Video Anomaly Detection [12.15221664801473]
  We introduce a patch-temporal framework for weakly supervised anomaly localization.<n>We propose a Proximitytemporal-Aware Topk selection strategy that enables the model to generate fine-grained spatial anomaly maps.<n>Our method surpasses existing state-the-art approaches across multiple benchmarks, yielding substantial gains in spatially supervised localization accuracy.
  arXiv  Detail & Related papers  (2026-06-28T16:52:29Z)
• AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE [51.994752158953084]
  AnchorMoE is an interpretable-by-construction classification framework.<n>It encodes multi-view representations of local patches and routes them to specialized experts.<n>Experiments on real-world and synthetic benchmarks demonstrate that AnchorMoE achieves highly competitive classification performance.
  arXiv  Detail & Related papers  (2026-06-02T13:30:54Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models [64.2820121997882]
  MIITA is a Memory-Induced Inference-Time Adaptation framework for supervised CL under constrained storage.<n>It stores supervised experiences as compact correction-direction prototypes with semantic anchors, and retrieves them at inference time using semantic and uncertainty-based cues.<n>Experiments show that MIITA consistently improves final performance and mitigates forgetting under fixed memory budgets.
  arXiv  Detail & Related papers  (2026-05-20T03:03:04Z)
• Next-Acceleration-Scale Prediction for Autoregressive MRI Reconstruction [52.32112533846212]
  MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions.<n>We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction.<n>We show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling.
  arXiv  Detail & Related papers  (2026-05-19T04:40:50Z)
• Word-Anchored Temporal Forgery Localization [6.691985085293349]
  We propose word-anchored temporal forgery localization (WAFL) to derive candidate forgery proposals.<n>We first analyze the essence of temporal forgeries and identify the minimum meaningful forgery units, word tokens, and then align data preprocessing with the natural linguistic boundaries of speech.<n>To overcome the extreme class imbalance inherent to forgery detection, we design the artifact-centric asymmetric (ACA) loss.
  arXiv  Detail & Related papers  (2026-03-06T12:33:28Z)
• GEM-TFL: Bridging Weak and Full Supervision for Forgery Localization through EM-Guided Decomposition and Temporal Refinement [24.929199892659636]
  Temporal Forgery Localization aims to precisely identify manipulated segments within videos or audio streams, providing interpretable evidence for multimedia forensics and security.<n>Most existing TFL methods rely on dense frame-level labels in a fully supervised manner, but Weakly Supervised TFL (WS-TFL) reduces labeling cost by learning only from binary video-level labels.<n>We propose GEM-TFL, a two-phase classification-regression framework that effectively bridges the supervision gap between training and inference.
  arXiv  Detail & Related papers  (2026-03-05T12:07:26Z)
• Escaping Spectral Bias without Backpropagation: Fast Implicit Neural Representations with Extreme Learning Machines [4.6894180050630005]
  Training implicit neural representations (INRs) to capture fine-scale details typically relies on iterative backpropagation.<n>We propose ELM-INR, a backpropagation-free INR that decomposes the domain into iterative and fits each local problem using an Extreme Learning Machine (ELM) in closed form.<n>This design yields fast and numerically robust reconstruction by combining local predictors through a partition of unity.
  arXiv  Detail & Related papers  (2026-02-07T16:12:39Z)
• From Passive Perception to Active Memory: A Weakly Supervised Image Manipulation Localization Framework Driven by Coarse-Grained Annotations [14.0185129202898]
  BoxPromptIML is a novel weakly-supervised IML framework that balances annotation cost and localization performance.<n>Inspired by the human subconscious memory mechanism, our feature fusion module employs a dual-guidance strategy that actively contextualizes recalled patterns with real-time observational cues.
  arXiv  Detail & Related papers  (2025-11-25T14:39:17Z)
• Glocal Information Bottleneck for Time Series Imputation [70.41814118117311]
  Time Series Imputation aims to recover missing values in temporal data.<n>Existing models typically optimize the point-wise reconstruction loss, focusing on recovering numerical values (local information)<n>We propose a new training paradigm, Glocal Information Bottleneck (Glocal-IB)
  arXiv  Detail & Related papers  (2025-10-06T15:24:44Z)
• PDE Solvers Should Be Local: Fast, Stable Rollouts with Learned Local Stencils [20.49015396991881]
  We present FINO, a finite-difference-inspired neural architecture that enforces strict locality.<n>FINO replaces fixed finite-difference stencil coefficients with learnable convolutional kernels.<n>It achieves up to 44% lower error and up to around 2times speedups over state-of-the-art operator-learning baselines.
  arXiv  Detail & Related papers  (2025-09-30T12:42:32Z)
• A Multimodal Deviation Perceiving Framework for Weakly-Supervised Temporal Forgery Localization [21.13433908232578]
  We present a framework for weakly-supervised temporal forgery localization.<n>It aims to identify temporal partial forged segments using only video-level annotations.<n>Extensive experiments demonstrate the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2025-07-22T13:55:16Z)
• Context-aware TFL: A Universal Context-aware Contrastive Learning Framework for Temporal Forgery Localization [60.73623588349311]
  We propose a universal context-aware contrastive learning framework (UniCaCLF) for temporal forgery localization.<n>Our approach leverages supervised contrastive learning to discover and identify forged instants by means of anomaly detection.<n>An efficient context-aware contrastive coding is introduced to further push the limit of instant feature distinguishability between genuine and forged instants.
  arXiv  Detail & Related papers  (2025-06-10T06:40:43Z)
• DGTR: Distributed Gaussian Turbo-Reconstruction for Sparse-View Vast Scenes [81.56206845824572]
  Novel-view synthesis (NVS) approaches play a critical role in vast scene reconstruction. Few-shot methods often struggle with poor reconstruction quality in vast environments. This paper presents DGTR, a novel distributed framework for efficient Gaussian reconstruction for sparse-view vast scenes.
  arXiv  Detail & Related papers  (2024-11-19T07:51:44Z)
• Subspace-Guided Feature Reconstruction for Unsupervised Anomaly Localization [5.085309164633571]
  Unsupervised anomaly localization plays a critical role in industrial manufacturing. Most recent methods perform feature matching or reconstruction for the target sample with pre-trained deep neural networks. We propose a novel subspace-guided feature reconstruction framework to pursue adaptive feature approximation for anomaly localization.
  arXiv  Detail & Related papers  (2023-09-25T06:58:57Z)
• Semi-Supervised Temporal Action Detection with Proposal-Free Masking [134.26292288193298]
  We propose a novel Semi-supervised Temporal action detection model based on PropOsal-free Temporal mask (SPOT) SPOT outperforms state-of-the-art alternatives, often by a large margin.
  arXiv  Detail & Related papers  (2022-07-14T16:58:47Z)
• Delving into Sequential Patches for Deepfake Detection [64.19468088546743]
  Recent advances in face forgery techniques produce nearly untraceable deepfake videos, which could be leveraged with malicious intentions. Previous studies has identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods. We propose the Local- & Temporal-aware Transformer-based Deepfake Detection framework, which adopts a local-to-global learning protocol.
  arXiv  Detail & Related papers  (2022-07-06T16:46:30Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.