LLDif: Diffusion Models for Low-light Emotion Recognition

• URL: http://arxiv.org/abs/2408.04235v1
• Date: Thu, 8 Aug 2024 05:41:09 GMT
• Title: LLDif: Diffusion Models for Low-light Emotion Recognition
• Authors: Zhifeng Wang, Kaihao Zhang, Ramesh Sankaranarayana,
• Abstract summary: This paper introduces LLDif, a novel diffusion-based facial expression recognition (FER) framework tailored for extremely low-light (LL) environments. Images captured under such conditions often suffer from low brightness and significantly reduced contrast, presenting challenges to conventional methods. LLDif addresses these issues with a novel two-stage training process that combines a Label-aware CLIP (LA-CLIP), an embedding prior network (PNET) and a transformer-based network adept at handling the noise of low-light images.
• Score: 15.095166627983566
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces LLDif, a novel diffusion-based facial expression recognition (FER) framework tailored for extremely low-light (LL) environments. Images captured under such conditions often suffer from low brightness and significantly reduced contrast, presenting challenges to conventional methods. These challenges include poor image quality that can significantly reduce the accuracy of emotion recognition. LLDif addresses these issues with a novel two-stage training process that combines a Label-aware CLIP (LA-CLIP), an embedding prior network (PNET), and a transformer-based network adept at handling the noise of low-light images. The first stage involves LA-CLIP generating a joint embedding prior distribution (EPD) to guide the LLformer in label recovery. In the second stage, the diffusion model (DM) refines the EPD inference, ultilising the compactness of EPD for precise predictions. Experimental evaluations on various LL-FER datasets have shown that LLDif achieves competitive performance, underscoring its potential to enhance FER applications in challenging lighting conditions.

Related papers

• Diffusion-Guided Knowledge Distillation for Weakly-Supervised Low-Light Semantic Segmentation [32.4542162399975]
  Weakly-supervised semantic segmentation aims to assign category labels to each pixel using weak annotations.<n>Existing methods significantly degrade in low-light environments due to severe image quality degradation and inherent constraints of weak supervision.<n>We propose Diffusion-Guided Knowledge Distillation for WeaklySupervised Low-light Semantic (DGKD-WLSS)
  arXiv  Detail & Related papers  (2025-07-10T09:28:54Z)
• One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation [53.24542646616045]
  We propose VPD-SR, a novel visual perception diffusion distillation framework specifically designed for image super-resolution (SR) generation.<n>VPD-SR consists of two components: Explicit Semantic-aware Supervision (ESS) and High-frequency Perception (HFP) loss.<n>The proposed VPD-SR achieves superior performance compared to both previous state-of-the-art methods and the teacher model with just one-step sampling.
  arXiv  Detail & Related papers  (2025-06-03T08:28:13Z)
• Unsupervised Low-light Image Enhancement with Lookup Tables and Diffusion Priors [38.96909959677438]
  Low-light image enhancement (LIE) aims at precisely and efficiently recovering an image degraded in poor illumination environments. Recent advanced LIE techniques are using deep neural networks, which require lots of low-normal light image pairs, network parameters, and computational resources. We devise a novel unsupervised LIE framework based on diffusion priors and lookup tables to achieve efficient low-light image recovery.
  arXiv  Detail & Related papers  (2024-09-27T16:37:27Z)
• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• Retinex-RAWMamba: Bridging Demosaicing and Denoising for Low-Light RAW Image Enhancement [71.13353154514418]
  Low-light image enhancement, particularly in cross-domain tasks such as mapping from the raw domain to the sRGB domain, remains a significant challenge.<n>We propose a novel Mamba-based method customized for low light RAW images, called RAWMamba, to effectively handle raw images with different CFAs.<n>By bridging demosaicing and denoising, better enhancement for low light RAW images is achieved.
  arXiv  Detail & Related papers  (2024-09-11T06:12:03Z)
• AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancement [37.274077278901494]
  We propose the Attribute Guidance Diffusion framework (AGLLDiff) for effective real-world LIE. AGLLDiff shifts the paradigm and models the desired attributes, such as image exposure, structure and color of normal-light images. Our approach outperforms the current leading unsupervised LIE methods across benchmarks in terms of distortion-based and perceptual-based metrics.
  arXiv  Detail & Related papers  (2024-07-20T15:17:48Z)
• CodeEnhance: A Codebook-Driven Approach for Low-Light Image Enhancement [97.95330185793358]
  Low-light image enhancement (LLIE) aims to improve low-illumination images. Existing methods face two challenges: uncertainty in restoration from diverse brightness degradations and loss of texture and color information. We propose a novel enhancement approach, CodeEnhance, by leveraging quantized priors and image refinement.
  arXiv  Detail & Related papers  (2024-04-08T07:34:39Z)
• LRDif: Diffusion Models for Under-Display Camera Emotion Recognition [16.965454529686177]
  This study introduces LRDif, a novel diffusion-based framework designed specifically for facial expression recognition (FER) To address the inherent challenges posed by UDC's image degradation, LRDif employs a two-stage training strategy that integrates a condensed preliminary extraction network (FPEN) and an agile transformer network (UDCformer)
  arXiv  Detail & Related papers  (2024-02-01T00:19:57Z)
• LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models [54.93010869546011]
  We propose to leverage the pre-trained latent diffusion model to perform the neural ISP for enhancing extremely low-light images. Specifically, to tailor the pre-trained latent diffusion model to operate on the RAW domain, we train a set of lightweight taming modules. We observe different roles of UNet denoising and decoder reconstruction in the latent diffusion model, which inspires us to decompose the low-light image enhancement task into latent-space low-frequency content generation and decoding-phase high-frequency detail maintenance.
  arXiv  Detail & Related papers  (2023-12-02T04:31:51Z)
• LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement [118.83316133601319]
  Current deep learning methods for low-light image enhancement (LLIE) typically rely on pixel-wise mapping learned from paired data. We propose a degradation-aware learning scheme for LLIE using diffusion models, which effectively integrates degradation and image priors into the diffusion process.
  arXiv  Detail & Related papers  (2023-07-27T07:22:51Z)
• Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition [78.50328335703914]
  Diffusion in the Dark (DiD) is a diffusion model for low-light image reconstruction for text recognition. We demonstrate that DiD, without any task-specific optimization, can outperform SOTA low-light methods in low-light text recognition on real images.
  arXiv  Detail & Related papers  (2023-03-07T23:52:51Z)
• Unsupervised Low-light Image Enhancement with Decoupled Networks [103.74355338972123]
  We learn a two-stage GAN-based framework to enhance the real-world low-light images in a fully unsupervised fashion. Our proposed method outperforms the state-of-the-art unsupervised image enhancement methods in terms of both illumination enhancement and noise reduction.
  arXiv  Detail & Related papers  (2020-05-06T13:37:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.