Let's Enhance: A Deep Learning Approach to Extreme Deblurring of Text Images

• URL: http://arxiv.org/abs/2211.10103v2
• Date: Sun, 23 Apr 2023 11:44:02 GMT
• Title: Let's Enhance: A Deep Learning Approach to Extreme Deblurring of Text Images
• Authors: Theophil Trippe and Martin Genzel and Jan Macdonald and Maximilian M\"arz
• Abstract summary: This work presents a novel deep-learning-based pipeline for the inverse problem of image deblurring. Our results build on our winning submission to the recent Helsinki Deblurring Challenge 2021, whose goal was to explore the limits of state-of-the-art deblurring algorithms.
• Score: 3.441021278275805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work presents a novel deep-learning-based pipeline for the inverse problem of image deblurring, leveraging augmentation and pre-training with synthetic data. Our results build on our winning submission to the recent Helsinki Deblur Challenge 2021, whose goal was to explore the limits of state-of-the-art deblurring algorithms in a real-world data setting. The task of the challenge was to deblur out-of-focus images of random text, thereby in a downstream task, maximizing an optical-character-recognition-based score function. A key step of our solution is the data-driven estimation of the physical forward model describing the blur process. This enables a stream of synthetic data, generating pairs of ground-truth and blurry images on-the-fly, which is used for an extensive augmentation of the small amount of challenge data provided. The actual deblurring pipeline consists of an approximate inversion of the radial lens distortion (determined by the estimated forward model) and a U-Net architecture, which is trained end-to-end. Our algorithm was the only one passing the hardest challenge level, achieving over $70\%$ character recognition accuracy. Our findings are well in line with the paradigm of data-centric machine learning, and we demonstrate its effectiveness in the context of inverse problems. Apart from a detailed presentation of our methodology, we also analyze the importance of several design choices in a series of ablation studies. The code of our challenge submission is available under https://github.com/theophil-trippe/HDC_TUBerlin_version_1.

Related papers

• Blind Image Deblurring with FFT-ReLU Sparsity Prior [1.179778723980276]
  Blind image deblurring is the process of recovering a sharp image from a blurred one without prior knowledge about the blur kernel. We introduce a method that leverages a prior which targets the blur kernel to achieve effective deblurring across a wide range of image types.
  arXiv  Detail & Related papers  (2024-06-12T15:51:39Z)
• InfRS: Incremental Few-Shot Object Detection in Remote Sensing Images [11.916941756499435]
  In this paper, we explore the intricate task of incremental few-shot object detection in remote sensing images. We introduce a pioneering fine-tuning-based technique, termed InfRS, designed to facilitate the incremental learning of novel classes. We develop a prototypical calibration strategy based on the Wasserstein distance to mitigate the catastrophic forgetting problem.
  arXiv  Detail & Related papers  (2024-05-18T13:39:50Z)
• Enhancing Digital Hologram Reconstruction Using Reverse-Attention Loss for Untrained Physics-Driven Deep Learning Models with Uncertain Distance [10.788482076164314]
  We present a pioneering approach to addressing the Autofocusing challenge in untrained deep-learning methods. Our method presents a significant reconstruction performance over rival methods. For example, the difference is less than 1dB in PSNR and 0.002 in SSIM for the target sample.
  arXiv  Detail & Related papers  (2024-01-11T01:30:46Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation [151.70234052015948]
  We propose a novel approach that encourages the optimization algorithm to seek a flat trajectory. We show that the weights trained on synthetic data are robust against the accumulated errors perturbations with the regularization towards the flat trajectory. Our method, called Flat Trajectory Distillation (FTD), is shown to boost the performance of gradient-matching methods by up to 4.7%.
  arXiv  Detail & Related papers  (2022-11-20T15:49:11Z)
• Near-Exact Recovery for Tomographic Inverse Problems via Deep Learning [3.441021278275805]
  We show that an iterative end-to-end network scheme enables reconstructions close to numerical precision. We also demonstrate our state-of-the-art performance on the open-access real-world dataset LoDoPaB CT.
  arXiv  Detail & Related papers  (2022-06-14T10:06:41Z)
• High-resolution Iterative Feedback Network for Camouflaged Object Detection [128.893782016078]
  Spotting camouflaged objects that are visually assimilated into the background is tricky for object detection algorithms. We aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries. We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner.
  arXiv  Detail & Related papers  (2022-03-22T11:20:21Z)
• Learning Co-segmentation by Segment Swapping for Retrieval and Discovery [67.6609943904996]
  The goal of this work is to efficiently identify visually similar patterns from a pair of images. We generate synthetic training pairs by selecting object segments in an image and copy-pasting them into another image. We show our approach provides clear improvements for artwork details retrieval on the Brueghel dataset.
  arXiv  Detail & Related papers  (2021-10-29T16:51:16Z)
• DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows [145.83812019515818]
  We propose DeFlow, a method for learning image degradations from unpaired data. We model the degradation process in the latent space of a shared flow-decoder network. We validate our DeFlow formulation on the task of joint image restoration and super-resolution.
  arXiv  Detail & Related papers  (2021-01-14T18:58:01Z)
• Phase Retrieval with Holography and Untrained Priors: Tackling the Challenges of Low-Photon Nanoscale Imaging [7.984370990908576]
  Phase retrieval is the inverse problem of recovering a signal from magnitude-only Fourier measurements. We introduce a dataset-free deep learning framework for holographic phase retrieval adapted to nanoscale challenges.
  arXiv  Detail & Related papers  (2020-12-14T10:15:07Z)
• Self-Supervised Linear Motion Deblurring [112.75317069916579]
  Deep convolutional neural networks are state-of-the-art for image deblurring. We present a differentiable reblur model for self-supervised motion deblurring. Our experiments demonstrate that self-supervised single image deblurring is really feasible.
  arXiv  Detail & Related papers  (2020-02-10T20:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.