InstructIR: High-Quality Image Restoration Following Human Instructions

• URL: http://arxiv.org/abs/2401.16468v5
• Date: Wed, 25 Sep 2024 20:29:36 GMT
• Title: InstructIR: High-Quality Image Restoration Following Human Instructions
• Authors: Marcos V. Conde, Gregor Geigle, Radu Timofte,
• Abstract summary: We present the first approach that uses human-written instructions to guide the image restoration model. Our method, InstructIR, achieves state-of-the-art results on several restoration tasks.
• Score: 61.1546287323136
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Image restoration is a fundamental problem that involves recovering a high-quality clean image from its degraded observation. All-In-One image restoration models can effectively restore images from various types and levels of degradation using degradation-specific information as prompts to guide the restoration model. In this work, we present the first approach that uses human-written instructions to guide the image restoration model. Given natural language prompts, our model can recover high-quality images from their degraded counterparts, considering multiple degradation types. Our method, InstructIR, achieves state-of-the-art results on several restoration tasks including image denoising, deraining, deblurring, dehazing, and (low-light) image enhancement. InstructIR improves +1dB over previous all-in-one restoration methods. Moreover, our dataset and results represent a novel benchmark for new research on text-guided image restoration and enhancement. Our code, datasets and models are available at: https://github.com/mv-lab/InstructIR

Related papers

• Preserving Old Memories in Vivid Detail: Human-Interactive Photo Restoration Framework [19.213916633152625]
  Photo restoration can improve the quality of outcomes, but it often comes at a high price in terms of cost and time for restoration. We present the AI-based photo restoration framework composed of multiple stages, where each stage is tailored to enhance and restore specific types of photo damage. We present a novel old photo restoration dataset because we lack a publicly available dataset for our evaluation.
  arXiv  Detail & Related papers  (2024-10-12T13:23:08Z)
• UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation [50.27688690379488]
  Existing unified methods treat multi-degradation image restoration as a multi-task learning problem. We propose a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning. Our framework leverages the pre-trained generative model as the shared component for multi-degradation restoration and transfers it to specific degradation image restoration tasks.
  arXiv  Detail & Related papers  (2024-09-30T11:16:56Z)
• Review Learning: Advancing All-in-One Ultra-High-Definition Image Restoration Training Method [7.487270862599671]
  We propose a new training paradigm for general image restoration models, which we name bfReview Learning. This approach begins with sequential training of an image restoration model on several degraded datasets, combined with a review mechanism. We design a lightweight all-purpose image restoration network that can efficiently reason about degraded images with 4K resolution on a single consumer-grade GPU.
  arXiv  Detail & Related papers  (2024-08-13T08:08:45Z)
• Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild [57.06779516541574]
  SUPIR (Scaling-UP Image Restoration) is a groundbreaking image restoration method that harnesses generative prior and the power of model scaling up. We collect a dataset comprising 20 million high-resolution, high-quality images for model training, each enriched with descriptive text annotations.
  arXiv  Detail & Related papers  (2024-01-24T17:58:07Z)
• Improving Image Restoration through Removing Degradations in Textual Representations [60.79045963573341]
  We introduce a new perspective for improving image restoration by removing degradation in the textual representations of a degraded image. To address the cross-modal assistance, we propose to map the degraded images into textual representations for removing the degradations. In particular, We ingeniously embed an image-to-text mapper and text restoration module into CLIP-equipped text-to-image models to generate the guidance.
  arXiv  Detail & Related papers  (2023-12-28T19:18:17Z)
• ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restoration [46.87227160492818]
  We present Degradation-aware Visual Prompts, which encode various types of image degradation into unified visual prompts. These degradation-aware prompts provide control over image processing and allow weighted combinations for customized image restoration. We then leverage degradation-aware visual prompts to establish a controllable universal model for image restoration.
  arXiv  Detail & Related papers  (2023-06-23T17:59:47Z)
• PromptIR: Prompting for All-in-One Blind Image Restoration [64.02374293256001]
  We present a prompt-based learning approach, PromptIR, for All-In-One image restoration. Our method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. PromptIR offers a generic and efficient plugin module with few lightweight prompts.
  arXiv  Detail & Related papers  (2023-06-22T17:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.