A unified Benchmark for Multi-Frame Image Restoration under Severe Refractive Warping

• URL: http://arxiv.org/abs/2605.05079v1
• Date: Wed, 06 May 2026 16:14:21 GMT
• Title: A unified Benchmark for Multi-Frame Image Restoration under Severe Refractive Warping
• Abstract summary: Video sequence capturing through refractive dynamic media, such as a turbulent air or water surface, often suffer from severe geometric distortions and temporal instability.<n>We present a comprehensive benchmark for geometric distortion removal in video, covering a range from turbulence-like mild warping to strong discontinuous refractive deformations.
• Score: 1.8935634168098605
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Video sequence capturing through refractive dynamic media, such as a turbulent air or water surface, often suffer from severe geometric distortions and temporal instability. While recent advances address mild atmospheric turbulence, no existing benchmarks systematically evaluate restoration methods under strong and highly nonuniform refractive conditions. We present a comprehensive benchmark for geometric distortion removal in video, covering a range from turbulence-like mild warping to strong discontinuous refractive deformations. The benchmark includes both laboratory-captured real data and synthetic sequences generated for static scenes via physics-based light refraction modeling across four distortion levels and multiple surface wave types. We evaluate a spectrum of methods from simple baselines and classical registration algorithms to advanced learning-based approaches including DATUM and our proposed diffusion based V-cache for high and extreme distortions regimes. Evaluation uses both pixel-level (PSNR, SSIM), and perceptual (LPIPS, DINO, CLIP) metrics providing the first large scale analysis of geometric distortion removal. Our benchmark establishes a new foundation for developing and evaluating algorithms capable of reconstructing video from highly distorted optical environments. Our code and datasets are available at https://github.com/iafoss/refractive-mfir-benchmark.

Related papers

• RefracGS: Novel View Synthesis Through Refractive Water Surfaces with 3D Gaussian Ray Tracing [59.24689613992402]
  We introduce RefracGS, a framework that jointly reconstructs the refractive water surface and the scene beneath the interface.<n>RefracGS outperforms prior refractive methods in visual quality, while achieving 15x faster training and real-time rendering at 200 FPS.
  arXiv  Detail & Related papers  (2026-03-23T08:31:08Z)
• VIGOR: VIdeo Geometry-Oriented Reward for Temporal Generative Alignment [15.619170225414571]
  Video diffusion models lack explicit geometric supervision during training, leading to inconsistency artifacts.<n>We propose a geometry-based reward model that leverages pretrained geometric foundation models to evaluate multi-view consistency.<n>Our approach conducts error computation in a pointwise fashion, yielding a more physically grounded and robust error metric.
  arXiv  Detail & Related papers  (2026-03-17T09:04:10Z)
• gQIR: Generative Quanta Image Reconstruction [18.400282448827507]
  We present an approach that adapts large text-to-image latent diffusion models to the photon-limited domain of quanta burst imaging.<n>By integrating latent-space restoration with burst-level-temporal reasoning, our approach produces reconstructions that are both photometrically faithful and perceptually pleasing.
  arXiv  Detail & Related papers  (2026-02-23T23:33:00Z)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference [83.38607296779423]
  We show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.<n>Our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
  arXiv  Detail & Related papers  (2025-10-13T13:29:20Z)
• GratNet: A Photorealistic Neural Shader for Diffractive Surfaces [0.0]
  We present a multi-layer perceptron (MLP) based method for data-driven rendering of diffractive surfaces with high accuracy and efficiency.<n>We demonstrate the high-quality reconstruction of the ground-truth using Peak-Signal-to-Noise (PSNR), Structural Similarity Index Measure (SSIM) and a flipping difference evaluator (FLIP) as evaluation metrics.
  arXiv  Detail & Related papers  (2025-06-18T18:58:00Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [80.20339155618612]
  DiffusionAD is a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network.<n>A rapid one-step denoising paradigm achieves hundreds of times acceleration while preserving comparable reconstruction quality.<n>Considering the diversity in the manifestation of anomalies, we propose a norm-guided paradigm to integrate the benefits of multiple noise scales.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• Making Reconstruction-based Method Great Again for Video Anomaly Detection [64.19326819088563]
  Anomaly detection in videos is a significant yet challenging problem. Existing reconstruction-based methods rely on old-fashioned convolutional autoencoders. We propose a new autoencoder model for enhanced consecutive frame reconstruction.
  arXiv  Detail & Related papers  (2023-01-28T01:57:57Z)
• Sampling Neural Radiance Fields for Refractive Objects [8.539183778516795]
  In this work, the scene is instead a heterogeneous volume with a piecewise-constant refractive index, where the path will be curved if it intersects the different refractive indices. For novel view synthesis of refractive objects, our NeRF-based framework aims to optimize the radiance fields of bounded volume and boundary from multi-view posed images with refractive object silhouettes. Given the refractive index, we extend the stratified and hierarchical sampling techniques in NeRF to allow drawing samples along a curved path tracked by the Eikonal equation.
  arXiv  Detail & Related papers  (2022-11-27T11:43:21Z)
• AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models [64.24948495708337]
  Atmospheric turbulence causes significant degradation to image quality by introducing blur and geometric distortion. Various deep learning-based single image atmospheric turbulence mitigation methods, including CNN-based and GAN inversion-based, have been proposed. Denoising Diffusion Probabilistic Models (DDPMs) have recently gained some traction because of their stable training process and their ability to generate high quality images.
  arXiv  Detail & Related papers  (2022-08-24T03:13:04Z)
• Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction [61.17654438176999]
  We propose a novel and robust unsupervised video anomaly detection method by frame prediction with proper design. Our proposed method obtains the frame-level AUROC score of 88.3% on the CUHK Avenue dataset.
  arXiv  Detail & Related papers  (2020-11-05T11:34:12Z)
• Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic Turbulence [1.6114012813668934]
  We present a unified method for atmospheric turbulence mitigation in both static and dynamic sequences. We are able to achieve better results compared to existing methods by utilizing a novel space-time non-local averaging method.
  arXiv  Detail & Related papers  (2020-08-31T19:20:46Z)
• Learning to Restore a Single Face Image Degraded by Atmospheric Turbulence using CNNs [93.72048616001064]
  Images captured under such condition suffer from a combination of geometric deformation and space varying blur. We present a deep learning-based solution to the problem of restoring a turbulence-degraded face image.
  arXiv  Detail & Related papers  (2020-07-16T15:25:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.