Continuous Facial Motion Deblurring

• URL: http://arxiv.org/abs/2207.06626v1
• Date: Thu, 14 Jul 2022 02:53:37 GMT
• Title: Continuous Facial Motion Deblurring
• Abstract summary: We propose a continuous facial motion deblurring network based on GAN (CFMD-GAN) To stabilize the network training, we train the generator to restore continuous moments in the order determined by our facial motion-based reordering process. Experiments on the 300VW datasets demonstrate that the proposed framework generates a various number of continuous output frames by varying the moment control factor.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce a novel framework for continuous facial motion deblurring that restores the continuous sharp moment latent in a single motion-blurred face image via a moment control factor. Although a motion-blurred image is the accumulated signal of continuous sharp moments during the exposure time, most existing single image deblurring approaches aim to restore a fixed number of frames using multiple networks and training stages. To address this problem, we propose a continuous facial motion deblurring network based on GAN (CFMD-GAN), which is a novel framework for restoring the continuous moment latent in a single motion-blurred face image with a single network and a single training stage. To stabilize the network training, we train the generator to restore continuous moments in the order determined by our facial motion-based reordering process (FMR) utilizing domain-specific knowledge of the face. Moreover, we propose an auxiliary regressor that helps our generator produce more accurate images by estimating continuous sharp moments. Furthermore, we introduce a control-adaptive (ContAda) block that performs spatially deformable convolution and channel-wise attention as a function of the control factor. Extensive experiments on the 300VW datasets demonstrate that the proposed framework generates a various number of continuous output frames by varying the moment control factor. Compared with the recent single-to-single image deblurring networks trained with the same 300VW training set, the proposed method show the superior performance in restoring the central sharp frame in terms of perceptual metrics, including LPIPS, FID and Arcface identity distance. The proposed method outperforms the existing single-to-video deblurring method for both qualitative and quantitative comparisons.

Related papers

• Eulerian Motion Guidance: Robust Image Animation via Bidirectional Geometric Consistency [74.90075313101933]
  This paper revisits the same optical-flow primitive through a more local supervision design.<n>We use adjacent-frame Eulerian motion fields to guide generation, where the motion signal always describes a short temporal hop.<n>To mitigate the drift artifacts common in adjacent frame generation, we introduce a Bidirectional Geometric Consistency mechanism.
  arXiv  Detail & Related papers  (2026-05-07T13:53:31Z)
• ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving [31.688411695647357]
  We propose ST-Prune, a training-free, plug-and-play framework comprising two complementary modules: Motionaware RSP (MTP) and Ring-view Spatial Pruning (RSPRSP)<n>These two modules constitute a complete-temporal pruning process, preserving key geometry-the-art for training-free token pruning.<n> ST-Prune achieves near-lossless performance with certain metrics surpassing the full-model baseline, while maintaining speeds comparable to existing pruning approaches.
  arXiv  Detail & Related papers  (2026-04-21T06:51:08Z)
• DiTVR: Zero-Shot Diffusion Transformer for Video Restoration [48.97196894658511]
  DiTVR is a zero shot video restoration framework that couples a diffusion transformer with trajectory aware attention and a flow consistent sampler.<n>Our attention mechanism aligns tokens along optical flow trajectories, with particular emphasis on vital layers that exhibit the highest sensitivity to temporal dynamics.<n>The flow guided sampler injects data consistency only into low-frequency bands, preserving high frequency priors while accelerating cache.
  arXiv  Detail & Related papers  (2025-08-11T09:54:45Z)
• Low-Light Video Enhancement via Spatial-Temporal Consistent Illumination and Reflection Decomposition [68.6707284662443]
  Low-Light Video Enhancement (LLVE) seeks to restore dynamic and static scenes plagued by severe invisibility and noise. One critical aspect is formulating a consistency constraint specifically for temporal-spatial illumination and appearance enhanced versions. We present an innovative video Retinex-based decomposition strategy that operates without the need for explicit supervision.
  arXiv  Detail & Related papers  (2024-05-24T15:56:40Z)
• Recovering Continuous Scene Dynamics from A Single Blurry Image with Events [58.7185835546638]
  An Implicit Video Function (IVF) is learned to represent a single motion blurred image with concurrent events. A dual attention transformer is proposed to efficiently leverage merits from both modalities. The proposed network is trained only with the supervision of ground-truth images of limited referenced timestamps.
  arXiv  Detail & Related papers  (2023-04-05T18:44:17Z)
• Modeling Continuous Motion for 3D Point Cloud Object Tracking [54.48716096286417]
  This paper presents a novel approach that views each tracklet as a continuous stream. At each timestamp, only the current frame is fed into the network to interact with multi-frame historical features stored in a memory bank. To enhance the utilization of multi-frame features for robust tracking, a contrastive sequence enhancement strategy is proposed.
  arXiv  Detail & Related papers  (2023-03-14T02:58:27Z)
• A Constrained Deformable Convolutional Network for Efficient Single Image Dynamic Scene Blind Deblurring with Spatially-Variant Motion Blur Kernels Estimation [12.744989551644744]
  We propose a novel constrained deformable convolutional network (CDCN) for efficient single image dynamic scene blind deblurring. CDCN simultaneously achieves accurate spatially-variant motion blur kernels estimation and the high-quality image restoration.
  arXiv  Detail & Related papers  (2022-08-23T03:28:21Z)
• Spatio-Temporal Transformer for Dynamic Facial Expression Recognition in the Wild [19.5702895176141]
  We propose a method for capturing discnative features within each frame model. We utilize the CNN to translate each frame into a visual feature sequence. Experiments indicate that our method provides an effective way to make use of the spatial and temporal dependencies.
  arXiv  Detail & Related papers  (2022-05-10T08:47:15Z)
• SL-CycleGAN: Blind Motion Deblurring in Cycles using Sparse Learning [3.6828576285640353]
  We introduce an end-to-end generative adversarial network (GAN) based on sparse learning for single image blind motion deblurring. For the first time in blind motion deblurring, we propose a sparse ResNet-block as a combination of sparse convolution layers and a trainable spatial pooler k-winner. We achieve the record-breaking PSNR of 38.087 dB on GoPro dataset, which is 5.377 dB better than the most recent deblurring method.
  arXiv  Detail & Related papers  (2021-11-07T07:42:22Z)
• Blind Motion Deblurring Super-Resolution: When Dynamic Spatio-Temporal Learning Meets Static Image Understanding [87.5799910153545]
  Single-image super-resolution (SR) and multi-frame SR are two ways to super resolve low-resolution images. Blind Motion Deblurring Super-Reslution Networks is proposed to learn dynamic-temporal information from single static motion-blurred images.
  arXiv  Detail & Related papers  (2021-05-27T11:52:45Z)
• Self-Supervised Multi-Frame Monocular Scene Flow [61.588808225321735]
  We introduce a multi-frame monocular scene flow network based on self-supervised learning. We observe state-of-the-art accuracy among monocular scene flow methods based on self-supervised learning.
  arXiv  Detail & Related papers  (2021-05-05T17:49:55Z)
• Restoration of Video Frames from a Single Blurred Image with Motion Understanding [69.90724075337194]
  We propose a novel framework to generate clean video frames from a single motion-red image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors. Our framework is based on anblur-decoder structure with spatial transformer network modules.
  arXiv  Detail & Related papers  (2021-04-19T08:32:57Z)
• Intrinsic Temporal Regularization for High-resolution Human Video Synthesis [59.54483950973432]
  temporal consistency is crucial for extending image processing pipelines to the video domain. We propose an effective intrinsic temporal regularization scheme, where an intrinsic confidence map is estimated via the frame generator to regulate motion estimation. We apply our intrinsic temporal regulation to single-image generator, leading to a powerful " INTERnet" capable of generating $512times512$ resolution human action videos.
  arXiv  Detail & Related papers  (2020-12-11T05:29:45Z)
• Deep Frequent Spatial Temporal Learning for Face Anti-Spoofing [9.435020319411311]
  Face anti-spoofing is crucial for the security of face recognition system, by avoiding invaded with presentation attack. Previous works have shown the effectiveness of using depth and temporal supervision for this task. We propose a novel two stream FreqSaptialTemporalNet for face anti-spoofing which simultaneously takes advantage of frequent, spatial and temporal information.
  arXiv  Detail & Related papers  (2020-01-20T06:02:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.