SL-CycleGAN: Blind Motion Deblurring in Cycles using Sparse Learning

• URL: http://arxiv.org/abs/2111.04026v1
• Date: Sun, 7 Nov 2021 07:42:22 GMT
• Title: SL-CycleGAN: Blind Motion Deblurring in Cycles using Sparse Learning
• Abstract summary: We introduce an end-to-end generative adversarial network (GAN) based on sparse learning for single image blind motion deblurring. For the first time in blind motion deblurring, we propose a sparse ResNet-block as a combination of sparse convolution layers and a trainable spatial pooler k-winner. We achieve the record-breaking PSNR of 38.087 dB on GoPro dataset, which is 5.377 dB better than the most recent deblurring method.
• Score: 3.6828576285640353
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we introduce an end-to-end generative adversarial network (GAN) based on sparse learning for single image blind motion deblurring, which we called SL-CycleGAN. For the first time in blind motion deblurring, we propose a sparse ResNet-block as a combination of sparse convolution layers and a trainable spatial pooler k-winner based on HTM (Hierarchical Temporal Memory) to replace non-linearity such as ReLU in the ResNet-block of SL-CycleGAN generators. Furthermore, unlike many state-of-the-art GAN-based motion deblurring methods that treat motion deblurring as a linear end-to-end process, we take our inspiration from the domain-to-domain translation ability of CycleGAN, and we show that image deblurring can be cycle-consistent while achieving the best qualitative results. Finally, we perform extensive experiments on popular image benchmarks both qualitatively and quantitatively and achieve the record-breaking PSNR of 38.087 dB on GoPro dataset, which is 5.377 dB better than the most recent deblurring method.

Related papers

• Optimally Bridging Semantics and Data: Generative Semantic Communication via Schrödinger Bridge [59.85230215774303]
  Generative Semantic Communication (GSC) is a promising solution for image transmission over narrow-band and high-noise channels.<n>Existing GSC methods rely on long, indirect transport trajectories from a Gaussian to an image distribution guided by semantics.<n>We propose Schrdinger Bridge-based GSC that breaks Gaussian limitations and enables direct generative decoding from semantics to images.
  arXiv  Detail & Related papers  (2026-04-20T04:49:58Z)
• Nesterov Method for Asynchronous Pipeline Parallel Optimization [59.79227116582264]
  We introduce a variant of Nesterov Accelerated Gradient (NAG) for asynchronous optimization in Pipeline Parallelism.<n>Specifically, we modify the look-ahead step in NAG to effectively address the staleness in gradients.<n>We theoretically prove that our approach converges at a sublinear rate in the presence of fixed delay in gradients.
  arXiv  Detail & Related papers  (2025-05-02T08:23:29Z)
• Rethinking the Upsampling Layer in Hyperspectral Image Super Resolution [51.98465973507002]
  We propose a novel lightweight SHSR network, i.e., LKCA-Net, that incorporates channel attention to calibrate multi-scale channel features of hyperspectral images. We demonstrate, for the first time, that the low-rank property of the learnable upsampling layer is a key bottleneck in lightweight SHSR methods.
  arXiv  Detail & Related papers  (2025-01-30T15:43:34Z)
• SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training [68.7896349660824]
  We present an in-depth analysis of the progressive overfitting problem from the lens of Seq FT. Considering that the overly fast representation learning and the biased classification layer constitute this particular problem, we introduce the advanced Slow Learner with Alignment (S++) framework. Our approach involves a Slow Learner to selectively reduce the learning rate of backbone parameters, and a Alignment to align the disjoint classification layers in a post-hoc fashion.
  arXiv  Detail & Related papers  (2024-08-15T17:50:07Z)
• Gyroscope-Assisted Motion Deblurring Network [11.404195533660717]
  This paper presents a framework to synthetic and restore motion blur images using Inertial Measurement Unit (IMU) data. The framework includes a strategy for training triplet generation, and a Gyroscope-Aided Motion Deblurring (GAMD) network for blurred image restoration.
  arXiv  Detail & Related papers  (2024-02-10T01:30:24Z)
• StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video Generation [73.54398908446906]
  We introduce a novel motion generator design that uses a learning-based inversion network for GAN. Our method supports style transfer with simple fine-tuning when the encoder is paired with a pretrained StyleGAN generator.
  arXiv  Detail & Related papers  (2023-08-31T17:59:33Z)
• NL-CS Net: Deep Learning with Non-Local Prior for Image Compressive Sensing [7.600617428107161]
  Deep learning has been applied to compressive sensing (CS) of images successfully in recent years. This paper proposes a novel CS method using non-local prior which combines the interpretability of the traditional optimization methods with the speed of network-based methods, called NL-CS Net.
  arXiv  Detail & Related papers  (2023-05-06T02:34:28Z)
• Continuous Facial Motion Deblurring [0.0]
  We propose a continuous facial motion deblurring network based on GAN (CFMD-GAN) To stabilize the network training, we train the generator to restore continuous moments in the order determined by our facial motion-based reordering process. Experiments on the 300VW datasets demonstrate that the proposed framework generates a various number of continuous output frames by varying the moment control factor.
  arXiv  Detail & Related papers  (2022-07-14T02:53:37Z)
• Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problem [97.64313409741614]
  We prove fast mixing and characterize the stationary distribution of the Langevin Algorithm for inverting random weighted DNN generators. We propose to do posterior sampling in the latent space of a pre-trained generative model.
  arXiv  Detail & Related papers  (2022-06-18T03:47:37Z)
• Orthogonalising gradients to speed up neural network optimisation [0.0]
  optimisation of neural networks can be sped up by orthogonalising the gradients before the optimisation step, ensuring the diversification of the learned representations. We tested this method on ImageNet and CIFAR-10 resulting in a large decrease in learning time, and also obtain a speed-up on the semi-supervised learning BarlowTwins.
  arXiv  Detail & Related papers  (2022-02-14T21:46:07Z)
• Optical-Flow-Reuse-Based Bidirectional Recurrent Network for Space-Time Video Super-Resolution [52.899234731501075]
  Space-time video super-resolution (ST-VSR) simultaneously increases the spatial resolution and frame rate for a given video. Existing methods typically suffer from difficulties in how to efficiently leverage information from a large range of neighboring frames. We propose a coarse-to-fine bidirectional recurrent neural network instead of using ConvLSTM to leverage knowledge between adjacent frames.
  arXiv  Detail & Related papers  (2021-10-13T15:21:30Z)
• Two-Stage Self-Supervised Cycle-Consistency Network for Reconstruction of Thin-Slice MR Images [62.4428833931443]
  The thick-slice magnetic resonance (MR) images are often structurally blurred in coronal and sagittal views. Deep learning has shown great potential to re-construct the high-resolution (HR) thin-slice MR images from those low-resolution (LR) cases. We propose a novel Two-stage Self-supervised Cycle-consistency Network (TSCNet) for MR slice reconstruction.
  arXiv  Detail & Related papers  (2021-06-29T13:29:18Z)
• On the spatial attention in Spatio-Temporal Graph Convolutional Networks for skeleton-based human action recognition [97.14064057840089]
  Graphal networks (GCNs) promising performance in skeleton-based human action recognition by modeling a sequence of skeletons as a graph. Most of the recently proposed G-temporal-based methods improve the performance by learning the graph structure at each layer of the network.
  arXiv  Detail & Related papers  (2020-11-07T19:03:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.