Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring

• URL: http://arxiv.org/abs/2004.05343v1
• Date: Sat, 11 Apr 2020 09:24:00 GMT
• Title: Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring
• Authors: Maitreya Suin, Kuldeep Purohit, A. N. Rajagopalan
• Abstract summary: We propose an efficient pixel adaptive and feature attentive design for handling large blur variations across different spatial locations. We use a patch-hierarchical attentive architecture composed of the above module that implicitly discovers the spatial variations in the blur present in the input image. Our design offers significant improvements over the state-of-the-art in accuracy as well as speed.
• Score: 39.92889091819711
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper tackles the problem of motion deblurring of dynamic scenes. Although end-to-end fully convolutional designs have recently advanced the state-of-the-art in non-uniform motion deblurring, their performance-complexity trade-off is still sub-optimal. Existing approaches achieve a large receptive field by increasing the number of generic convolution layers and kernel-size, but this comes at the expense of of the increase in model size and inference speed. In this work, we propose an efficient pixel adaptive and feature attentive design for handling large blur variations across different spatial locations and process each test image adaptively. We also propose an effective content-aware global-local filtering module that significantly improves performance by considering not only global dependencies but also by dynamically exploiting neighbouring pixel information. We use a patch-hierarchical attentive architecture composed of the above module that implicitly discovers the spatial variations in the blur present in the input image and in turn, performs local and global modulation of intermediate features. Extensive qualitative and quantitative comparisons with prior art on deblurring benchmarks demonstrate that our design offers significant improvements over the state-of-the-art in accuracy as well as speed.

Related papers

• LADDER: An Efficient Framework for Video Frame Interpolation [12.039193291203492]
  Video Frame Interpolation (VFI) is a crucial technique in various applications such as slow-motion generation, frame rate conversion, video frame restoration etc. This paper introduces an efficient video frame framework that aims to strike a favorable balance between efficiency and quality.
  arXiv  Detail & Related papers  (2024-04-17T06:47:17Z)
• Spatially-Attentive Patch-Hierarchical Network with Adaptive Sampling for Motion Deblurring [34.751361664891235]
  We propose a pixel adaptive and feature attentive design for handling large blur variations across different spatial locations. We show that our approach performs favorably against the state-of-the-art deblurring algorithms.
  arXiv  Detail & Related papers  (2024-02-09T01:00:09Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks [88.77951448313486]
  We present a new approach for model acceleration by exploiting spatial sparsity in visual data. We propose a dynamic token sparsification framework to prune redundant tokens. We extend our method to hierarchical models including CNNs and hierarchical vision Transformers.
  arXiv  Detail & Related papers  (2022-07-04T17:00:51Z)
• Motion Estimation for Large Displacements and Deformations [7.99536002595393]
  Variational optical flow techniques based on a coarse-to-fine scheme interpolate sparse matches and locally optimize an energy model conditioned on colour, gradient and smoothness. This paper addresses this problem and presents HybridFlow, a variational motion estimation framework for large displacements and deformations.
  arXiv  Detail & Related papers  (2022-06-24T18:53:22Z)
• CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training [112.96224800952724]
  We propose cascaded modulation GAN (CM-GAN) to generate plausible image structures when dealing with large holes in complex images. In each decoder block, global modulation is first applied to perform coarse semantic-aware synthesis structure, then spatial modulation is applied on the output of global modulation to further adjust the feature map in a spatially adaptive fashion. In addition, we design an object-aware training scheme to prevent the network from hallucinating new objects inside holes, fulfilling the needs of object removal tasks in real-world scenarios.
  arXiv  Detail & Related papers  (2022-03-22T16:13:27Z)
• Adaptive Single Image Deblurring [43.02281823557039]
  We propose an efficient pixel adaptive and feature attentive design for handling large blur variations within and across different images. We also propose an effective content-aware global-local filtering module that significantly improves the performance.
  arXiv  Detail & Related papers  (2022-01-01T10:10:19Z)
• EAN: Event Adaptive Network for Enhanced Action Recognition [66.81780707955852]
  We propose a unified action recognition framework to investigate the dynamic nature of video content. First, when extracting local cues, we generate the spatial-temporal kernels of dynamic-scale to adaptively fit the diverse events. Second, to accurately aggregate these cues into a global video representation, we propose to mine the interactions only among a few selected foreground objects by a Transformer.
  arXiv  Detail & Related papers  (2021-07-22T15:57:18Z)
• Algorithm-hardware Co-design for Deformable Convolution [40.50544352625659]
  We build an efficient object detection network with modified deformable convolutions and quantize the network using state-of-the-art quantization methods. Preliminary experiments show that little accuracy is compromised and speedup can be achieved with our co-design optimization for the deformable convolution.
  arXiv  Detail & Related papers  (2020-02-19T01:08:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.