Disentangle Nighttime Lens Flares: Self-supervised Generation-based Lens Flare Removal

• URL: http://arxiv.org/abs/2502.10714v1
• Date: Sat, 15 Feb 2025 08:04:38 GMT
• Title: Disentangle Nighttime Lens Flares: Self-supervised Generation-based Lens Flare Removal
• Authors: Yuwen He, Wei Wang, Wanyu Wang, Kui Jiang,
• Abstract summary: Lens flares arise from light reflection and refraction within sensor arrays, whose diverse types include glow, veiling glare, reflective flare and so on. Existing methods are specialized for one specific type only, and overlook the simultaneous occurrence of multiple typed lens flares. We introduce a solution named Self-supervised Generation-based Lens Flare Removal Network (SGLFR-Net), which is self-supervised without pre-training.
• Score: 18.825840100537174
• License:
• Abstract: Lens flares arise from light reflection and refraction within sensor arrays, whose diverse types include glow, veiling glare, reflective flare and so on. Existing methods are specialized for one specific type only, and overlook the simultaneous occurrence of multiple typed lens flares, which is common in the real-world, e.g. coexistence of glow and displacement reflections from the same light source. These co-occurring lens flares cannot be effectively resolved by the simple combination of individual flare removal methods, since these coexisting flares originates from the same light source and are generated simultaneously within the same sensor array, exhibit a complex interdependence rather than simple additive relation. To model this interdependent flare relationship, our Nighttime Lens Flare Formation model is the first attempt to learn the intrinsic physical relationship between flares on the imaging plane. Building on this physical model, we introduce a solution to this joint flare removal task named Self-supervised Generation-based Lens Flare Removal Network (SGLFR-Net), which is self-supervised without pre-training. Specifically, the nighttime glow is detangled in PSF Rendering Network(PSFR-Net) based on PSF Rendering Prior, while the reflective flare is modelled in Texture Prior Based Reflection Flare Removal Network (TPRR-Net). Empirical evaluations demonstrate the effectiveness of the proposed method in both joint and individual glare removal tasks.

Related papers

• GN-FR:Generalizable Neural Radiance Fields for Flare Removal [11.646119341279164]
  We propose a framework to render flare-free views from a sparse set of input images affected by lens flare. We present a 3D multi-view flare dataset, comprising 17 real flare scenes with 782 images, 80 real flare patterns, and their corresponding annotated flare-occupancy masks.
  arXiv  Detail & Related papers  (2024-12-11T08:43:52Z)
• Difflare: Removing Image Lens Flare with Latent Diffusion Model [19.022105366814078]
  We introduce Difflare, a novel approach designed for lens flare removal. To leverage the generative prior learned by Pre-Trained Diffusion Models (PTDM), we introduce a trainable Structural Guidance Injection Module (SGIM) To address information loss resulting from latent compression, we introduce an Adaptive Feature Fusion Module (AFFM)
  arXiv  Detail & Related papers  (2024-07-20T04:36:39Z)
• Improving Lens Flare Removal with General Purpose Pipeline and Multiple Light Sources Recovery [69.71080926778413]
  flare artifacts can affect image visual quality and downstream computer vision tasks. Current methods do not consider automatic exposure and tone mapping in image signal processing pipeline. We propose a solution to improve the performance of lens flare removal by revisiting the ISP and design a more reliable light sources recovery strategy.
  arXiv  Detail & Related papers  (2023-08-31T04:58:17Z)
• Toward Real Flare Removal: A Comprehensive Pipeline and A New Benchmark [12.1632995709273]
  We propose a well-developed methodology for generating data-pairs with flare deterioration. The similarity of scattered flares and symmetric effect of reflected ghosts are realized. We also construct a real-shot pipeline that respectively processes the effects of scattering and reflective flares.
  arXiv  Detail & Related papers  (2023-06-28T02:57:25Z)
• Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond [77.72043833102191]
  We introduce the first comprehensive nighttime flare removal dataset, consisting of 962 real-captured flare images (Flare-R) and 7,000 synthetic flares (Flare7K) Compared to Flare7K, Flare7K++ is particularly effective in eliminating complicated degradation around the light source, which is intractable by using synthetic flares alone. To address this issue, we additionally provide the annotations of light sources in Flare7K++ and propose a new end-to-end pipeline to preserve the light source while removing lens flares.
  arXiv  Detail & Related papers  (2023-06-07T08:27:44Z)
• Relightify: Relightable 3D Faces from a Single Image via Diffusion Models [86.3927548091627]
  We present the first approach to use diffusion models as a prior for highly accurate 3D facial BRDF reconstruction from a single image. In contrast to existing methods, we directly acquire the observed texture from the input image, thus, resulting in more faithful and consistent estimation.
  arXiv  Detail & Related papers  (2023-05-10T11:57:49Z)
• Nighttime Smartphone Reflective Flare Removal Using Optical Center Symmetry Prior [81.64647648269889]
  Reflective flare is a phenomenon that occurs when light reflects inside lenses, causing bright spots or a "ghosting effect" in photos. We propose an optical center symmetry prior, which suggests that the reflective flare and light source are always symmetrical around the lens's optical center. We create the first reflective flare removal dataset called BracketFlare, which contains diverse and realistic reflective flare patterns.
  arXiv  Detail & Related papers  (2023-03-27T09:44:40Z)
• Flare7K: A Phenomenological Nighttime Flare Removal Dataset [83.38205781536578]
  We introduce Flare7K, the first nighttime flare removal dataset. It offers 5,000 scattering and 2,000 reflective flare images, consisting of 25 types of scattering flares and 10 types of reflective flares. With the paired data, we can train deep models to restore flare-corrupted images taken in the real world effectively.
  arXiv  Detail & Related papers  (2022-10-12T20:17:24Z)
• How to Train Neural Networks for Flare Removal [45.51943926089249]
  We train neural networks to remove lens flare for the first time. Our data synthesis approach is critical for accurate flare removal. Models trained with our technique generalize well to real lens flares across different scenes, lighting conditions, and cameras.
  arXiv  Detail & Related papers  (2020-11-25T02:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.