ExpoMamba: Exploiting Frequency SSM Blocks for Efficient and Effective Image Enhancement

• URL: http://arxiv.org/abs/2408.09650v1
• Date: Mon, 19 Aug 2024 02:16:47 GMT
• Title: ExpoMamba: Exploiting Frequency SSM Blocks for Efficient and Effective Image Enhancement
• Authors: Eashan Adhikarla, Kai Zhang, John Nicholson, Brian D. Davison,
• Abstract summary: ExpoMamba is a novel architecture that integrates components of the frequency state space within a modified U-Net. Our experiments demonstrate that ExpoMamba enhances low-light images up to 2-3x faster than traditional models.
• Score: 7.091012207482573
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Low-light image enhancement remains a challenging task in computer vision, with existing state-of-the-art models often limited by hardware constraints and computational inefficiencies, particularly in handling high-resolution images. Recent foundation models, such as transformers and diffusion models, despite their efficacy in various domains, are limited in use on edge devices due to their computational complexity and slow inference times. We introduce ExpoMamba, a novel architecture that integrates components of the frequency state space within a modified U-Net, offering a blend of efficiency and effectiveness. This model is specifically optimized to address mixed exposure challenges, a common issue in low-light image enhancement, while ensuring computational efficiency. Our experiments demonstrate that ExpoMamba enhances low-light images up to 2-3x faster than traditional models with an inference time of 36.6 ms and achieves a PSNR improvement of approximately 15-20% over competing models, making it highly suitable for real-time image processing applications.

Related papers

• Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis [62.06970466554273]
  We present Meissonic, which non-autoregressive masked image modeling (MIM) text-to-image elevates to a level comparable with state-of-the-art diffusion models like SDXL. We leverage high-quality training data, integrate micro-conditions informed by human preference scores, and employ feature compression layers to further enhance image fidelity and resolution. Our model not only matches but often exceeds the performance of existing models like SDXL in generating high-quality, high-resolution images.
  arXiv  Detail & Related papers  (2024-10-10T17:59:17Z)
• Expansive Supervision for Neural Radiance Field [12.510474224361504]
  We introduce an expansive supervision mechanism that efficiently balances computational load, rendering quality and flexibility for neural radiance field training. Compared to conventional supervision, our method effectively bypasses redundant rendering processes, resulting in notable reductions in both time and memory consumption.
  arXiv  Detail & Related papers  (2024-09-12T14:05:13Z)
• Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models [26.926712014346432]
  This paper presents innovative enhancements to diffusion models by integrating a novel multi-resolution network and time-dependent layer normalization. Our method's efficacy is demonstrated on the class-conditional ImageNet generation benchmark, setting new state-of-the-art FID scores of 1.70 on ImageNet 256 x 256 and 2.89 on ImageNet 512 x 512.
  arXiv  Detail & Related papers  (2024-06-13T17:59:58Z)
• Scaling Diffusion Mamba with Bidirectional SSMs for Efficient Image and Video Generation [41.54814517077309]
  We introduce a novel diffusion architecture, Diffusion Mamba (DiM), which foregoes traditional attention mechanisms in favor of a scalable alternative. DiM achieves rapid inference times and reduced computational load, maintaining linear complexity with respect to sequence length. Results affirm the scalability and efficiency of DiM, establishing a new benchmark for image and video generation techniques.
  arXiv  Detail & Related papers  (2024-05-24T18:50:27Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis [56.849285913695184]
  Diffusion Mamba (DiM) is a sequence model for efficient high-resolution image synthesis. DiM architecture achieves inference-time efficiency for high-resolution images. Experiments demonstrate the effectiveness and efficiency of our DiM.
  arXiv  Detail & Related papers  (2024-05-23T06:53:18Z)
• DARK: Denoising, Amplification, Restoration Kit [0.7670170505111058]
  This paper introduces a novel lightweight computational framework for enhancing images under low-light conditions. Our model is designed to be lightweight, ensuring low computational demand and suitability for real-time applications on standard consumer hardware.
  arXiv  Detail & Related papers  (2024-05-21T16:01:13Z)
• SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions [5.100085108873068]
  We present two models, SDXS-512 and SDXS-1024, achieving inference speeds of approximately 100 FPS (30x faster than SD v1.5) and 30 FPS (60x faster than SDXL) on a single GPU. Our training approach offers promising applications in image-conditioned control, facilitating efficient image-to-image translation.
  arXiv  Detail & Related papers  (2024-03-25T11:16:23Z)
• E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation [69.72194342962615]
  We introduce and address a novel research direction: can the process of distilling GANs from diffusion models be made significantly more efficient? First, we construct a base GAN model with generalized features, adaptable to different concepts through fine-tuning, eliminating the need for training from scratch. Second, we identify crucial layers within the base GAN model and employ Low-Rank Adaptation (LoRA) with a simple yet effective rank search process, rather than fine-tuning the entire base model. Third, we investigate the minimal amount of data necessary for fine-tuning, further reducing the overall training time.
  arXiv  Detail & Related papers  (2024-01-11T18:59:14Z)
• Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day [63.96075838322437]
  We propose a framework to learn a single-image novel-view synthesizer. Our framework is able to reduce the total training time from 10 days to less than 1 day.
  arXiv  Detail & Related papers  (2023-10-04T17:57:07Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.