Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake Localization

• URL: http://arxiv.org/abs/2605.23113v1
• Date: Fri, 22 May 2026 00:17:16 GMT
• Title: Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake Localization
• Abstract summary: IaMSB is an inconsistency-aware multimodal Schrdinger Bridge.<n>It unifies consistency estimation, cross-modal information selection, and bridge-step scheduling.<n>It stabilizes strict-IoU boundary precision, raising AP@0.95 by 3%10%, and yields improved high-precision localization.
• Score: 11.418398950934671
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Audio-visual deepfake localization demands interval-level outputs that serve as temporal evidence. Despite recent progress, symmetric fusion under single-sided or asynchronous forgeries propagates cross-modal noise, degrading high-precision localization. We present IaMSB, an inconsistency-aware multimodal Schrödinger Bridge (SB) that jointly estimates cross-modal consistency and performs interval-level localization. Unlike diffusion models, SB minimizes path-distribution discrepancy and yields consistency scores without explicit noise injection or denoising. With the Schrödinger Bridge (SB), IaMSB unifies consistency estimation, cross-modal information selection, and bridge-step scheduling in one framework. Specifically, a lightweight coarse bridge first proposes candidate intervals and estimates cross-modal consistency; these statistics select cross-modal witness signals and allocate bridge steps asymmetrically across modalities. A refinement bridge then performs step-tuned fusion and outputs refined, time-aligned intervals. IaMSB anticipates single-sided and asynchronous forgeries and, using bottlenecked cross-modal interaction with step allocation, suppresses noise transfer, avoids unnecessary iterations. Across benchmarks, IaMSB stabilizes strict-IoU boundary precision, raising AP@0.95 by 3%~10%, and yields improved high-precision localization, particularly for single-sided forgeries.

Related papers

• Reliable Near-Field Multi-User Positioning Informed by Two-Stage MUSIC [52.33021019321273]
  Near-field localization is a promising technique for high-resolution multi-user positioning in future wireless systems.<n>Existing near-field localization methods, which require separate parameter estimation and path/source association, suffer from high overhead and accumulated errors.<n>We propose an end-to-end near-field positioning deep learning (DL) framework informed by two-stage MUltiple SIgnal Classification (MUSIC) in mixed line-of-sight (LoS) and non-LoS (NLoS) multi-path scenarios.
  arXiv  Detail & Related papers  (2026-09-08T20:03:33Z)
• S$^3$AM: A Single-Stream SAM with Reliability-Calibrated Frequency Adapter for Multi-modal Salient Object Detection [74.15663758681849]
  Single-stream framework integrates reliability-calibrated frequency adaptation into the adopted SAM backbone for MSOD.<n>Experiments on RGB-D, RGB-T, and RGB-NIR salient object detection benchmarks validate that the proposed framework achieves competitive performance.
  arXiv  Detail & Related papers  (2026-08-18T08:05:00Z)
• HybridSB-MoE: Dual-Domain Schrödinger Bridges with Scene-Adaptive Expert Routing for Speech Enhancement [8.444837651455275]
  HybridSB-MoE is a dual-domain framework for generative speech enhancement.<n>It fills three gaps: spectral models capture harmonic structure but often disrupt phase, waveform models preserve phase but miss harmonics, and Schrdinger Bridges (SB) shorten transport from noise to clean speech but leave cost only loosely tied to training.<n>On VoiceBank+DEMAND, HybridSB-MoE outperforms diffusion- and SB-based baselines at their step budgets.
  arXiv  Detail & Related papers  (2026-08-13T02:00:52Z)
• Twisted Schrödinger Bridge Matching [34.126987447337534]
  Diffusion-based Schrdinger bridge models have been proposed to approximate optimal transport dynamics between two prescribed boundary distributions.<n>We introduce Twisted Schrdinger Bridge Matching, a diffusion-based method designed to handle both continuous- and discrete-time potentials.
  arXiv  Detail & Related papers  (2026-07-18T22:30:43Z)
• OmniForcing: Unleashing Real-time Joint Audio-Visual Generation [51.031726911410594]
  Recent joint audio-visual diffusion models achieve remarkable generation quality but suffer from high latency.<n>We propose OmniForcing, the first framework to distill an offline, dual-stream bidirectional diffusion model into a high-fidelity streaming autore generator.
  arXiv  Detail & Related papers  (2026-03-12T08:17:36Z)
• Learning to Separate RF Signals Under Uncertainty: Detect-Then-Separate vs. Unified Joint Models [53.79667447811139]
  We show that a single deep neural architecture learns to jointly detect and separate when applied directly to the received signal.<n>These findings highlight UJM as a scalable and practical alternative to DTS, while opening new directions for unified separation under broader estimation.
  arXiv  Detail & Related papers  (2026-02-04T15:25:02Z)
• Neighbor-aware Instance Refining with Noisy Labels for Cross-Modal Retrieval [12.062625455647265]
  Cross-Modal Retrieval (CMR) has made significant progress in the field of multi-modal analysis.<n> CMR methods often fail to simultaneously satisfy model performance ceilings, calibration reliability, and data utilization rate.<n>We propose a novel robust cross-modal learning framework, namely Neighbor-aware Instance Refining with Noisy Labels (NIRNL)
  arXiv  Detail & Related papers  (2025-12-30T08:19:07Z)
• Cross-Attention with Confidence Weighting for Multi-Channel Audio Alignment [5.380078543698624]
  Multi-channel audio alignment is a key requirement in bioacoustic monitoring, spatial audio systems, and acoustic localization.<n>We introduce a method that combines cross-attention mechanisms with confidence-weighted scoring to improve multi-channel audio synchronization.<n>Our method achieved first place in the BioDCASE 2025 Task 1 challenge with 0.30 MSE average across test datasets, compared to 0.58 for the deep learning baseline.
  arXiv  Detail & Related papers  (2025-09-21T05:14:06Z)
• Momentum Multi-Marginal Schrödinger Bridge Matching [20.53138254276056]
  We introduce textbfMomentum Multi-Marginal Schr"odinger Bridge Matching (3MSBM), a novel matching framework that learns smooth measure-valued splines for systems that satisfy multiple positional constraints.<n>As a matching approach, 3MSBM learns transport maps that preserve intermediate marginals throughout training, significantly improving convergence and scalability.
  arXiv  Detail & Related papers  (2025-06-11T20:41:23Z)
• Branched Schrödinger Bridge Matching [45.105452288011726]
  We introduce Branched Schr"odinger Bridge Matching (BranchSBM), a novel framework that learns branched Schr"odinger bridges.<n>BranchSBM parameterizes multiple time-dependent velocity fields and growth processes, enabling the representation of population-level divergence into terminal distributions.<n>We show that BranchSBM is not only more expressive but also essential for tasks involving multi-path surface navigation, modeling cell fate bifurcations from homogeneous progenitor states, and simulating diverging cellular responses to perturbations.
  arXiv  Detail & Related papers  (2025-06-10T17:29:48Z)
• Dequantified Diffusion-Schr{ö}dinger Bridge for Density Ratio Estimation [15.067770053726534]
  Density ratio estimation is fundamental to tasks involving $f$-divergences.<n>We propose a unified framework for textbfrobust, textbfstable and textbfefficient density ratio estimation.
  arXiv  Detail & Related papers  (2025-05-08T08:12:16Z)
• Single-Step Bidirectional Unpaired Image Translation Using Implicit Bridge Consistency Distillation [55.45188329646137]
  Implicit Bridge Consistency Distillation (IBCD) enables single-step bidirectional unpaired translation without using adversarial loss.<n>IBCD achieves state-of-the-art performance on benchmark datasets in a single generation step.
  arXiv  Detail & Related papers  (2025-03-19T09:48:04Z)
• Diffusion Bridge Implicit Models [25.213664260896103]
  Denoising diffusion bridge models (DDBMs) are a powerful variant of diffusion models for interpolating between two arbitrary paired distributions.<n>We take the first step in fast sampling of DDBMs without extra training, motivated by the well-established recipes in diffusion models.<n>We induce a novel, simple, and insightful form of ordinary differential equation (ODE) which inspires high-order numerical solvers.
  arXiv  Detail & Related papers  (2024-05-24T19:08:30Z)
• Adversarial Schrödinger Bridge Matching [66.39774923893103]
  Iterative Markovian Fitting (IMF) procedure alternates between Markovian and reciprocal projections of continuous-time processes. We propose a novel Discrete-time IMF (D-IMF) procedure in which learning of processes is replaced by learning just a few transition probabilities in discrete time. We show that our D-IMF procedure can provide the same quality of unpaired domain translation as the IMF, using only several generation steps instead of hundreds.
  arXiv  Detail & Related papers  (2024-05-23T11:29:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.