Prompt-Guided Image-Adaptive Neural Implicit Lookup Tables for Interpretable Image Enhancement

• URL: http://arxiv.org/abs/2408.11055v1
• Date: Tue, 20 Aug 2024 17:59:01 GMT
• Title: Prompt-Guided Image-Adaptive Neural Implicit Lookup Tables for Interpretable Image Enhancement
• Authors: Satoshi Kosugi,
• Abstract summary: interpretable image enhancement is a technique that enhances image quality by adjusting filter parameters with easily understandable names such as "Exposure" and "Contrast" We introduce an image-adaptive neural implicit lookup table, which uses a multilayer perceptron to implicitly define the transformation from input feature space to output color space. We evaluate visual impressions of enhancement results, such as exposure and contrast, using a vision and language model along with guiding prompts.
• Score: 4.233370898095789
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we delve into the concept of interpretable image enhancement, a technique that enhances image quality by adjusting filter parameters with easily understandable names such as "Exposure" and "Contrast". Unlike using predefined image editing filters, our framework utilizes learnable filters that acquire interpretable names through training. Our contribution is two-fold. Firstly, we introduce a novel filter architecture called an image-adaptive neural implicit lookup table, which uses a multilayer perceptron to implicitly define the transformation from input feature space to output color space. By incorporating image-adaptive parameters directly into the input features, we achieve highly expressive filters. Secondly, we introduce a prompt guidance loss to assign interpretable names to each filter. We evaluate visual impressions of enhancement results, such as exposure and contrast, using a vision and language model along with guiding prompts. We define a constraint to ensure that each filter affects only the targeted visual impression without influencing other attributes, which allows us to obtain the desired filter effects. Experimental results show that our method outperforms existing predefined filter-based methods, thanks to the filters optimized to predict target results. Our source code is available at https://github.com/satoshi-kosugi/PG-IA-NILUT.

Related papers

• ERUP-YOLO: Enhancing Object Detection Robustness for Adverse Weather Condition by Unified Image-Adaptive Processing [0.5312303275762104]
  We propose an image-adaptive object detection method for adverse weather conditions such as fog and low-light. Our framework employs differentiable preprocessing filters to perform image enhancement suitable for later-stage object detections. We evaluate our proposed approach, called Enhanced Robustness by Unified Image Processing (ERUP)-YOLO, by applying it to the YOLOv3 detector.
  arXiv  Detail & Related papers  (2024-11-05T04:20:06Z)
• Enhancing Low-Light Images Using Infrared-Encoded Images [81.8710581927427]
  Previous arts mainly focus on the low-light images captured in the visible spectrum using pixel-wise loss. We propose a novel approach to increase the visibility of images captured under low-light environments by removing the in-camera infrared (IR) cut-off filter.
  arXiv  Detail & Related papers  (2023-07-09T08:29:19Z)
• Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler [103.97487121678276]
  Filter pruning simultaneously accelerates the computation and reduces the memory overhead of CNNs. We propose a novel Knowledge-driven Differential Filter Sampler(KDFS) with Masked Filter Modeling(MFM) framework for filter pruning.
  arXiv  Detail & Related papers  (2023-07-01T02:28:41Z)
• When A Conventional Filter Meets Deep Learning: Basis Composition Learning on Image Filters [20.506636435344333]
  We propose basis composition learning on single image filters to automatically determine their optimal formulas. Our method is simple yet effective in practice; it renders filters to be user-friendly and benefits fundamental low-level vision problems.
  arXiv  Detail & Related papers  (2022-03-01T06:34:54Z)
• Unsharp Mask Guided Filtering [53.14430987860308]
  The goal of this paper is guided image filtering, which emphasizes the importance of structure transfer during filtering. We propose a new and simplified formulation of the guided filter inspired by unsharp masking. Our formulation enjoys a filtering prior to a low-pass filter and enables explicit structure transfer by estimating a single coefficient.
  arXiv  Detail & Related papers  (2021-06-02T19:15:34Z)
• Instagram Filter Removal on Fashionable Images [2.1485350418225244]
  We introduce Instagram Filter Removal Network (IFRNet) to mitigate the effects of image filters for social media analysis applications. Experiments demonstrate IFRNet outperforms all compared methods in quantitative and qualitative comparisons. We present the filter classification performance of our proposed model, and analyze the dominant color estimation on the images unfiltered by all compared methods.
  arXiv  Detail & Related papers  (2021-04-11T18:44:43Z)
• Adaptive Debanding Filter [55.42929350861115]
  Banding artifacts manifest as staircase-like color bands on pictures or video frames. We propose a content-adaptive smoothing filtering followed by dithered quantization, as a post-processing module. Experimental results show that our proposed debanding filter outperforms state-of-the-art false contour removing algorithms both visually and quantitatively.
  arXiv  Detail & Related papers  (2020-09-22T20:44:20Z)
• Improved Adaptive Type-2 Fuzzy Filter with Exclusively Two Fuzzy Membership Function for Filtering Salt and Pepper Noise [30.639740354770282]
  fuzzy filter is proposed for filtering salt and pepper noise from the images. The proposed filter is validated on standard images with various noise levels. The performance of the proposed filter is compared with the various state-of-the-art methods in terms of peak signal-to-noise ratio and computation time.
  arXiv  Detail & Related papers  (2020-08-10T13:18:42Z)
• Filter Grafting for Deep Neural Networks: Reason, Method, and Cultivation [86.91324735966766]
  Filter is the key component in modern convolutional neural networks (CNNs) In this paper, we introduce filter grafting (textbfMethod) to achieve this goal. We develop a novel criterion to measure the information of filters and an adaptive weighting strategy to balance the grafted information among networks.
  arXiv  Detail & Related papers  (2020-04-26T08:36:26Z)
• Recognizing Instagram Filtered Images with Feature De-stylization [81.38905784617089]
  This paper presents a study on how popular pretrained models are affected by commonly used Instagram filters. Our analysis suggests that simple structure preserving filters which only alter the global appearance of an image can lead to large differences in the convolutional feature space. We introduce a lightweight de-stylization module that predicts parameters used for scaling and shifting feature maps to "undo" the changes incurred by filters.
  arXiv  Detail & Related papers  (2019-12-30T16:48:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.