Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement

• URL: http://arxiv.org/abs/2311.11695v1
• Date: Mon, 20 Nov 2023 11:51:13 GMT
• Title: Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement
• Authors: Yanyan Wei, Zhao Zhang, Jiahuan Ren, Xiaogang Xu, Richang Hong, Yi Yang, Shuicheng Yan, Meng Wang
• Abstract summary: We propose a transformative system that combines the conversational intelligence of ChatGPT with multiple IRE methods. Our case studies demonstrate that Clarity ChatGPT effectively improves the generalization and interaction capabilities in the IRE.
• Score: 97.41630939425731
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The generalization capability of existing image restoration and enhancement (IRE) methods is constrained by the limited pre-trained datasets, making it difficult to handle agnostic inputs such as different degradation levels and scenarios beyond their design scopes. Moreover, they are not equipped with interactive mechanisms to consider user preferences or feedback, and their end-to-end settings cannot provide users with more choices. Faced with the above-mentioned IRE method's limited performance and insufficient interactivity, we try to solve it from the engineering and system framework levels. Specifically, we propose Clarity ChatGPT-a transformative system that combines the conversational intelligence of ChatGPT with multiple IRE methods. Clarity ChatGPT can automatically detect image degradation types and select appropriate IRE methods to restore images, or iteratively generate satisfactory results based on user feedback. Its innovative features include a CLIP-powered detector for accurate degradation classification, no-reference image quality evaluation for performance evaluation, region-specific processing for precise enhancements, and advanced fusion techniques for optimal restoration results. Clarity ChatGPT marks a significant advancement in integrating language and vision, enhancing image-text interactions, and providing a robust, high-performance IRE solution. Our case studies demonstrate that Clarity ChatGPT effectively improves the generalization and interaction capabilities in the IRE, and also fills the gap in the low-level domain of the existing vision-language model.

Related papers

• AffectSRNet : Facial Emotion-Aware Super-Resolution Network [5.295131292624206]
  We propose AffectSRNet, a novel emotion-aware super-resolution framework for facial expression recognition. Our method bridges the gap between image resolution and expression accuracy by employing an expression-preserving loss function. We show that AffectSRNet outperforms existing FSR approaches in both visual quality and emotion fidelity.
  arXiv  Detail & Related papers  (2025-02-14T06:02:59Z)
• Adaptive Prompt Tuning: Vision Guided Prompt Tuning with Cross-Attention for Fine-Grained Few-Shot Learning [5.242869847419834]
  Few-shot, fine-grained classification in computer vision poses significant challenges due to the need to differentiate subtle class distinctions with limited data. This paper presents a novel method that enhances the Contrastive Language-Image Pre-Training model through adaptive prompt tuning.
  arXiv  Detail & Related papers  (2024-12-19T08:51:01Z)
• CLIP-SR: Collaborative Linguistic and Image Processing for Super-Resolution [21.843398350371867]
  Convolutional Neural Networks (CNNs) have advanced Image Super-Resolution (SR) Most CNN-based methods rely solely on pixel-based transformations, leading to artifacts and blurring. We introduce a multi-modal semantic enhancement approach that combines textual semantics with visual features.
  arXiv  Detail & Related papers  (2024-12-16T09:50:09Z)
• DemosaicFormer: Coarse-to-Fine Demosaicing Network for HybridEVS Camera [70.28702677370879]
  Hybrid Event-Based Vision Sensor (HybridEVS) is a novel sensor integrating traditional frame-based and event-based sensors. Despite its potential, the lack of Image signal processing (ISP) pipeline specifically designed for HybridEVS poses a significant challenge. We propose a coarse-to-fine framework named DemosaicFormer which comprises coarse demosaicing and pixel correction.
  arXiv  Detail & Related papers  (2024-06-12T07:20:46Z)
• SPIRE: Semantic Prompt-Driven Image Restoration [66.26165625929747]
  We develop SPIRE, a Semantic and restoration Prompt-driven Image Restoration framework. Our approach is the first framework that supports fine-level instruction through language-based quantitative specification of the restoration strength. Our experiments demonstrate the superior restoration performance of SPIRE compared to the state of the arts.
  arXiv  Detail & Related papers  (2023-12-18T17:02:30Z)
• Enhancement by Your Aesthetic: An Intelligible Unsupervised Personalized Enhancer for Low-Light Images [67.14410374622699]
  We propose an intelligible unsupervised personalized enhancer (iUPEnhancer) for low-light images. The proposed iUP-Enhancer is trained with the guidance of these correlations and the corresponding unsupervised loss functions. Experiments demonstrate that the proposed algorithm produces competitive qualitative and quantitative results.
  arXiv  Detail & Related papers  (2022-07-15T07:16:10Z)
• Controllable Image Enhancement [66.18525728881711]
  We present a semiautomatic image enhancement algorithm that can generate high-quality images with multiple styles by controlling a few parameters. An encoder-decoder framework encodes the retouching skills into latent codes and decodes them into the parameters of image signal processing functions.
  arXiv  Detail & Related papers  (2022-06-16T23:54:53Z)
• Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network [92.01145655155374]
  We present an unsupervised image enhancement generative network (UEGAN) It learns the corresponding image-to-image mapping from a set of images with desired characteristics in an unsupervised manner. Results show that the proposed model effectively improves the aesthetic quality of images.
  arXiv  Detail & Related papers  (2020-12-30T03:22:46Z)
• Image-Based Benchmarking and Visualization for Large-Scale Global Optimization [6.5447678518952115]
  An image-based visualization framework is proposed that visualizes the solutions to large-scale global optimization problems as images are proposed. In the proposed framework, the pixels visualize decision variables while the entire image represents the overall solution quality. The proposed framework is then demonstrated on arbitrary benchmark problems with known optima.
  arXiv  Detail & Related papers  (2020-07-24T03:39:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.