Towards Blind Bitstream-corrupted Video Recovery via a Visual Foundation Model-driven Framework

• URL: http://arxiv.org/abs/2507.22481v1
• Date: Wed, 30 Jul 2025 08:31:54 GMT
• Title: Towards Blind Bitstream-corrupted Video Recovery via a Visual Foundation Model-driven Framework
• Abstract summary: bitstream-corrupted video recovery has emerged as a challenging and understudied task.<n>We propose first blind bitstream-corrupted video recovery framework that integrates visual foundation models with a recovery model.<n>Our method achieves outstanding performance in bitstream-corrupted video recovery without requiring a manually labeled mask sequence.
• Score: 28.383016293452652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video signals are vulnerable in multimedia communication and storage systems, as even slight bitstream-domain corruption can lead to significant pixel-domain degradation. To recover faithful spatio-temporal content from corrupted inputs, bitstream-corrupted video recovery has recently emerged as a challenging and understudied task. However, existing methods require time-consuming and labor-intensive annotation of corrupted regions for each corrupted video frame, resulting in a large workload in practice. In addition, high-quality recovery remains difficult as part of the local residual information in corrupted frames may mislead feature completion and successive content recovery. In this paper, we propose the first blind bitstream-corrupted video recovery framework that integrates visual foundation models with a recovery model, which is adapted to different types of corruption and bitstream-level prompts. Within the framework, the proposed Detect Any Corruption (DAC) model leverages the rich priors of the visual foundation model while incorporating bitstream and corruption knowledge to enhance corruption localization and blind recovery. Additionally, we introduce a novel Corruption-aware Feature Completion (CFC) module, which adaptively processes residual contributions based on high-level corruption understanding. With VFM-guided hierarchical feature augmentation and high-level coordination in a mixture-of-residual-experts (MoRE) structure, our method suppresses artifacts and enhances informative residuals. Comprehensive evaluations show that the proposed method achieves outstanding performance in bitstream-corrupted video recovery without requiring a manually labeled mask sequence. The demonstrated effectiveness will help to realize improved user experience, wider application scenarios, and more reliable multimedia communication and storage systems.

Related papers

• Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal [56.34780145346335]
  Degradation-Aware Cross-Modal Prompt Compensation Network (DCMPC-Net)<n>DCMPC-Net consists of the Cross-Modal Prompt Generator (CMPG), Prompt-Guided Attention Alignment Module (PGAAM), and Dual Feature Compensation Module (DFCM)<n> experiments show that DCMPC-Net outperforms state-of-the-art methods in both task-specific and unified settings.
  arXiv  Detail & Related papers  (2026-08-07T08:13:40Z)
• DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration [31.441586261696617]
  This paper proposes DVANet, a collaborative unfolding process between degradation-aware observation consistency and visual-prior-guided reconstruction.<n>Experiments demonstrate that DVANet achieves superior or competitive performance on multi-scenario degradation and cross-domain image restoration tasks.
  arXiv  Detail & Related papers  (2026-06-17T14:09:39Z)
• SoLAR: Error-Resilient Streamable Long-Horizon Free-Viewpoint Video Reconstruction with Anchor Activation and Latent Recalibration [57.159190580279585]
  Motivated by bit allocation theory, we analyze dynamic-anchor-based volumetric video representation within a rate-distortion optimization framework.<n>We propose textbfSoLAR, which is the first error-resilient streamable FVV framework that maintains stable reconstruction quality on long sequences.
  arXiv  Detail & Related papers  (2026-05-08T06:48:59Z)
• Blind Bitstream-corrupted Video Recovery via Metadata-guided Diffusion Model [21.33395654187404]
  We introduce a new blind video recovery setting that removes the reliance on predefined masks.<n>We propose a Metadata-Guided Diffusion Model (M-GDM) to tackle these challenges.
  arXiv  Detail & Related papers  (2026-04-15T14:15:07Z)
• NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration: Methods and Results [106.34700405808155]
  NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration (BSCVR)<n>This paper reports on the NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration (BSCVR)
  arXiv  Detail & Related papers  (2026-04-08T11:08:55Z)
• All-in-One Video Restoration under Smoothly Evolving Unknown Weather Degradations [102.94052335735326]
  All-in-one image restoration aims to recover clean images from diverse unknown degradations using a single model.<n>Existing approaches primarily focus on frame-wise degradation variation, overlooking the temporal continuity that naturally exists in real-world degradation processes.<n>We introduce the Smoothly Evolving Unknown Degradations (SEUD) scenario, where both the active degradation set and degradation intensity change continuously over time.
  arXiv  Detail & Related papers  (2026-01-02T02:20:57Z)
• HaineiFRDM: Explore Diffusion to Restore Defects in Fast-Movement Films [2.3374825727995328]
  Existing open-source film restoration methods show limited performance compared to commercial methods.<n>We propose HaineiFRDM, a film restoration framework, to explore diffusion model's powerful content-understanding ability.
  arXiv  Detail & Related papers  (2025-12-31T16:18:07Z)
• MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration [62.929029990341796]
  Real-world videos often suffer from complex degradations, such as noise, compression artifacts, and low-light distortions.<n>We propose MoA-VR, which mimics the reasoning and processing procedures of human professionals through three coordinated agents.<n>Specifically, we construct a large-scale and high-resolution video degradation recognition benchmark and build a vision-language model (VLM) driven degradation identifier.
  arXiv  Detail & Related papers  (2025-10-09T17:42:51Z)
• LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration [3.2944592608677614]
  We propose LVTINO, the first zero-shot or plug-and-play inverse solver for high definition video restoration with priors encoded by VCMs.<n>Our conditioning mechanism bypasses the need for automatic differentiation and achieves state-of-the-art video reconstruction quality with only a few neural function evaluations.
  arXiv  Detail & Related papers  (2025-10-01T18:10:08Z)
• AR2: Attention-Guided Repair for the Robustness of CNNs Against Common Corruptions [5.294455344248843]
  Deep neural networks suffer from significant performance degradation when exposed to common corruptions.<n>We propose AR2 (Attention-Guided Repair for Robustness) to enhance the corruption robustness of pretrained CNNs.
  arXiv  Detail & Related papers  (2025-07-08T18:37:00Z)
• Temporal-Consistent Video Restoration with Pre-trained Diffusion Models [51.47188802535954]
  Video restoration (VR) aims to recover high-quality videos from degraded ones.<n>Recent zero-shot VR methods using pre-trained diffusion models (DMs) suffer from approximation errors during reverse diffusion and insufficient temporal consistency.<n>We present a novel a Posterior Maximum (MAP) framework that directly parameterizes video frames in the seed space of DMs, eliminating approximation errors.
  arXiv  Detail & Related papers  (2025-03-19T03:41:56Z)
• Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning [92.16191092329765]
  We investigate the impact of corrupted data on Multimodal Large Language Models (MLLMs)<n>We find that, although corrupted data degrade model performance, such adverse effects are largely reversible.<n>We introduce a corruption-robust training paradigm that significantly surpasses existing strategies for mitigating the effects of corrupted data.
  arXiv  Detail & Related papers  (2025-02-18T08:28:29Z)
• Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration [19.87693298262894]
  We propose Diff-Restorer, a universal image restoration method based on the diffusion model. We utilize the pre-trained visual language model to extract visual prompts from degraded images. We also design a Degradation-aware Decoder to perform structural correction and convert the latent code to the pixel domain.
  arXiv  Detail & Related papers  (2024-07-04T05:01:10Z)
• Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrieval [67.52910255064762]
  We design a simple dual-stream structure, including a temporal layer and a hash layer. We first design a simple dual-stream structure, including a temporal layer and a hash layer. With the help of semantic similarity knowledge obtained from self-supervision, the hash layer learns to capture information for semantic retrieval. In this way, the model naturally preserves the disentangled semantics into binary codes.
  arXiv  Detail & Related papers  (2023-10-12T03:21:12Z)
• Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Method [24.986639795808564]
  bitstream-corrupted video (BSCV) benchmark is first benchmark dataset with more than 28,000 video clips. BSCV is a collection of 1) a proposed three- parameter corruption model for video bitstream, 2) a large-scale dataset containing rich error patterns, multiple corruption levels, and flexible dataset branches, and 3) a plug-and-play module in video recovery framework that serves as a benchmark. We evaluate state-of-the-art video inpainting methods on the BSCV dataset, demonstrating existing approaches' limitations and our framework's advantages in solving the bitstream-corrupted video recovery problem
  arXiv  Detail & Related papers  (2023-09-25T06:06:26Z)
• Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration [78.14941737723501]
  We propose a Cross-consistent Deep Unfolding Network (CDUN) for All-In-One VR. By orchestrating two cascading procedures, CDUN achieves adaptive processing for diverse degradations. In addition, we introduce a window-based inter-frame fusion strategy to utilize information from more adjacent frames.
  arXiv  Detail & Related papers  (2023-09-04T14:18:00Z)
• Hierarchical Contrastive Learning for Pattern-Generalizable Image Corruption Detection [40.04083743934034]
  We develop a hierarchical contrastive learning framework to detect corrupted regions. A specialized hierarchical interaction mechanism is designed to facilitate the knowledge of contrastive learning in different scales. Our model has well generalization ability across different corruption patterns.
  arXiv  Detail & Related papers  (2023-08-27T10:03:48Z)
• PromptIR: Prompting for All-in-One Blind Image Restoration [64.02374293256001]
  We present a prompt-based learning approach, PromptIR, for All-In-One image restoration. Our method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. PromptIR offers a generic and efficient plugin module with few lightweight prompts.
  arXiv  Detail & Related papers  (2023-06-22T17:59:52Z)
• Making Reconstruction-based Method Great Again for Video Anomaly Detection [64.19326819088563]
  Anomaly detection in videos is a significant yet challenging problem. Existing reconstruction-based methods rely on old-fashioned convolutional autoencoders. We propose a new autoencoder model for enhanced consecutive frame reconstruction.
  arXiv  Detail & Related papers  (2023-01-28T01:57:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.