Blind Bitstream-corrupted Video Recovery via Metadata-guided Diffusion Model

• URL: http://arxiv.org/abs/2604.13906v1
• Date: Wed, 15 Apr 2026 14:15:07 GMT
• Title: Blind Bitstream-corrupted Video Recovery via Metadata-guided Diffusion Model
• Abstract summary: We introduce a new blind video recovery setting that removes the reliance on predefined masks.<n>We propose a Metadata-Guided Diffusion Model (M-GDM) to tackle these challenges.
• Score: 21.33395654187404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bitstream-corrupted video recovery aims to restore realistic content degraded during video storage or transmission. Existing methods typically assume that predefined masks of corrupted regions are available, but manually annotating these masks is labor-intensive and impractical in real-world scenarios. To address this limitation, we introduce a new blind video recovery setting that removes the reliance on predefined masks. This setting presents two major challenges: accurately identifying corrupted regions and recovering content from extensive and irregular degradations. We propose a Metadata-Guided Diffusion Model (M-GDM) to tackle these challenges. Specifically, intrinsic video metadata are leveraged as corruption indicators through a dual-stream metadata encoder that separately embeds motion vectors and frame types before fusing them into a unified representation. This representation interacts with corrupted latent features via cross-attention at each diffusion step. To preserve intact regions, we design a prior-driven mask predictor that generates pseudo masks using both metadata and diffusion priors, enabling the separation and recombination of intact and recovered regions through hard masking. To mitigate boundary artifacts caused by imperfect masks, a post-refinement module enhances consistency between intact and recovered regions. Extensive experiments demonstrate the effectiveness of our method and its superiority in blind video recovery. Code is available at: https://github.com/Shuyun-Wang/M-GDM.

Related papers

• Learning Unified Video and Image Representation for Video Face Forgery Detection [49.19035628283472]
  Existing methods for video face forgery detection assume that all frames in a forged video are manipulated.<n>We propose a novel framework, UVIF, that utilizes additional annotated images to provide fine-grained supervision for detecting partial forgeries in videos.<n>UVIF employs a unified encoder and a multi-task learning paradigm to jointly model facial videos and images for boosted video face forgery detection.
  arXiv  Detail & Related papers  (2026-08-13T10:25:59Z)
• SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer [4.205766319033597]
  We present SEDiT, a novel one-stage video Subtitle Erasure approach via One-step Diffusion Transformer.<n>We introduce a mask-free inference approach that enables direct erasure of the targeted subtitle.<n>Thanks to one-step and chunk-wise streaming inference, our method can efficiently handle native 1440p video with infinite length.
  arXiv  Detail & Related papers  (2026-05-14T14:37:54Z)
• ExposeAnyone: Personalized Audio-to-Expression Diffusion Models Are Robust Zero-Shot Face Forgery Detectors [58.45131932883374]
  We propose a fully self-supervised approach to detect deepfakes in videos.<n>Our model computes the identity distances between suspected videos and personalized subjects via diffusion reconstruction errors.<n>Our method is highly robust to corruptions such as blur and compression, highlighting the applicability in real-world face forgery detection.
  arXiv  Detail & Related papers  (2026-01-05T18:59:54Z)
• Backdoor Attacks on Prompt-Driven Video Segmentation Foundation Models [21.15174669411218]
  We propose BadVSFM, the first backdoor framework tailored to prompt-driven VSFMs.<n>BadVSFM uses a two-stage strategy: (1) steer the image encoder so triggered frames map to a designated target embedding while clean frames remain aligned with a clean reference encoder.<n>Experiments show that BadVSFM achieves strong, controllable backdoor effects under diverse triggers and prompts.
  arXiv  Detail & Related papers  (2025-12-26T14:48:58Z)
• DirectSwap: Mask-Free Cross-Identity Training and Benchmarking for Expression-Consistent Video Head Swapping [58.2549561389375]
  Video head swapping aims to replace the entire head of a video subject, including facial identity, head shape, and hairstyle, with that of a reference image.<n>Due to the lack of ground-truth paired swapping data, prior methods typically train on cross-frame pairs of the same person within a video.<n>We propose DirectSwap, a mask-free, direct video head-swapping framework that extends an image U-Net into a video diffusion model.
  arXiv  Detail & Related papers  (2025-12-10T08:31:28Z)
• Towards Blind Bitstream-corrupted Video Recovery via a Visual Foundation Model-driven Framework [28.383016293452652]
  bitstream-corrupted video recovery has emerged as a challenging and understudied task.<n>We propose first blind bitstream-corrupted video recovery framework that integrates visual foundation models with a recovery model.<n>Our method achieves outstanding performance in bitstream-corrupted video recovery without requiring a manually labeled mask sequence.
  arXiv  Detail & Related papers  (2025-07-30T08:31:54Z)
• DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion [94.46904504076124]
  Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. We introduce DiffusionFake, a novel framework that reverses the generative process of face forgeries to enhance the generalization of detection models.
  arXiv  Detail & Related papers  (2024-10-06T06:22:43Z)
• Unified Auto-Encoding with Masked Diffusion [15.264296748357157]
  We propose a unified self-supervised objective, dubbed Unified Masked Diffusion (UMD) UMD combines patch-based and noise-based corruption techniques within a single auto-encoding framework. It achieves strong performance in downstream generative and representation learning tasks.
  arXiv  Detail & Related papers  (2024-06-25T16:24:34Z)
• Siamese Masked Autoencoders [76.35448665609998]
  We present Siamese Masked Autoencoders (SiamMAE) for learning visual correspondence from videos. SiamMAE operates on pairs of randomly sampled video frames and asymmetrically masks them. It outperforms state-of-the-art self-supervised methods on video object segmentation, pose keypoint propagation, and semantic part propagation tasks.
  arXiv  Detail & Related papers  (2023-05-23T17:59:46Z)
• Mask to reconstruct: Cooperative Semantics Completion for Video-text Retrieval [19.61947785487129]
  Mask for Semantics Completion (MASCOT) based on semantic-based masked modeling. Our MASCOT performs state-of-the-art performance on four major text-video retrieval benchmarks.
  arXiv  Detail & Related papers  (2023-05-13T12:31:37Z)
• Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection [122.4894940892536]
  We present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss.
  arXiv  Detail & Related papers  (2022-09-25T04:56:10Z)
• Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness [66.55719330810547]
  Inpainting arbitrary missing regions is challenging because learning valid features for various masked regions is nontrivial. We propose a novel mask-aware inpainting solution that learns multi-scale features for missing regions in the encoding phase. Our framework is validated both quantitatively and qualitatively via extensive experiments on three public datasets.
  arXiv  Detail & Related papers  (2021-04-28T13:17:47Z)
• OLED: One-Class Learned Encoder-Decoder Network with Adversarial Context Masking for Novelty Detection [1.933681537640272]
  novelty detection is the task of recognizing samples that do not belong to the distribution of the target class. Deep autoencoders have been widely used as a base of many unsupervised novelty detection methods. We have designed a framework consisting of two competing networks, a Mask Module and a Reconstructor.
  arXiv  Detail & Related papers  (2021-03-27T17:59:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.