Recaptured Raw Screen Image and Video Demoir\'eing via Channel and Spatial Modulations

• URL: http://arxiv.org/abs/2310.20332v1
• Date: Tue, 31 Oct 2023 10:19:28 GMT
• Title: Recaptured Raw Screen Image and Video Demoir\'eing via Channel and Spatial Modulations
• Authors: Huanjing Yue and Yijia Cheng and Xin Liu and Jingyu Yang
• Abstract summary: We propose an image and video demoir'eing network tailored for raw inputs. We introduce a color-separated feature branch, and it is fused with the traditional feature-mixed branch via channel and spatial modulations. Experiments demonstrate that our method achieves state-of-the-art performance for both image and video demori'eing.
• Score: 16.122531943812465
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Capturing screen contents by smartphone cameras has become a common way for information sharing. However, these images and videos are often degraded by moir\'e patterns, which are caused by frequency aliasing between the camera filter array and digital display grids. We observe that the moir\'e patterns in raw domain is simpler than those in sRGB domain, and the moir\'e patterns in raw color channels have different properties. Therefore, we propose an image and video demoir\'eing network tailored for raw inputs. We introduce a color-separated feature branch, and it is fused with the traditional feature-mixed branch via channel and spatial modulations. Specifically, the channel modulation utilizes modulated color-separated features to enhance the color-mixed features. The spatial modulation utilizes the feature with large receptive field to modulate the feature with small receptive field. In addition, we build the first well-aligned raw video demoir\'eing (RawVDemoir\'e) dataset and propose an efficient temporal alignment method by inserting alternating patterns. Experiments demonstrate that our method achieves state-of-the-art performance for both image and video demori\'eing. We have released the code and dataset in https://github.com/tju-chengyijia/VD_raw.

Related papers

• Freqformer: Image-Demoiréing Transformer via Efficient Frequency Decomposition [83.40450475728792]
  We present Freqformer, a Transformer-based framework specifically designed for image demoir'eing through targeted frequency separation.<n>Our method performs an effective frequency decomposition that explicitly splits moir'e patterns into high-frequency spatially-localized textures and low-frequency scale-robust color distortions.<n>Experiments on various demoir'eing benchmarks demonstrate that Freqformer achieves state-of-the-art performance with a compact model size.
  arXiv  Detail & Related papers  (2025-05-25T12:23:10Z)
• DSDNet: Raw Domain Demoiréing via Dual Color-Space Synergy [17.598942972989228]
  We propose a single-stage raw domain demoir'eing framework, Dual-Stream Demoir'eing Network (DSDNet) To guide luminance correction and moir'e removal, we design a raw-to-YCbCr mapping pipeline. We also develop a Luminance-Chrominance Adaptive Transformer (LCAT) to better guide color fidelity.
  arXiv  Detail & Related papers  (2025-04-22T10:09:33Z)
• Color Matching Using Hypernetwork-Based Kolmogorov-Arnold Networks [44.97307414849601]
  cmKAN is a versatile framework for color matching. We use Kolmogorov-Arnold Networks (KANs) to model the color matching between source and target distributions. We introduce a first large-scale dataset of paired images captured by two distinct cameras.
  arXiv  Detail & Related papers  (2025-03-14T18:17:19Z)
• ViDSOD-100: A New Dataset and a Baseline Model for RGB-D Video Salient Object Detection [51.16181295385818]
  We first collect an annotated RGB-D video SODOD (DSOD-100) dataset, which contains 100 videos within a total of 9,362 frames. All the frames in each video are manually annotated to a high-quality saliency annotation. We propose a new baseline model, named attentive triple-fusion network (ATF-Net) for RGB-D salient object detection.
  arXiv  Detail & Related papers  (2024-06-18T12:09:43Z)
• Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers [18.731717752379232]
  Multi-Channel Imaging (MCI) models must support a variety of channel configurations at test time. Recent work has extended traditional visual encoders for MCI, such as Vision Transformers (ViT), by supplementing pixel information with an encoding representing the channel configuration. We propose DiChaViT, which aims to enhance the diversity in the learned features of MCI-ViT models.
  arXiv  Detail & Related papers  (2024-05-26T03:41:40Z)
• Color Image Denoising Using The Green Channel Prior [5.117362801192093]
  Green channel prior (GCP) is often understated or ignored in color image denoising. We propose a simple and effective one step GCP-based image denoising (GCP-ID) method.
  arXiv  Detail & Related papers  (2024-02-13T05:57:37Z)
• Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain [27.1716081216131]
  Current methods ignore the difference between cell phone RAW images and DSLR camera RGB images. We present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization.
  arXiv  Detail & Related papers  (2024-01-04T09:18:31Z)
• Training Neural Networks on RAW and HDR Images for Restoration Tasks [59.41340420564656]
  In this work, we test approaches on three popular image restoration applications: denoising, deblurring, and single-image super-resolution. Our results indicate that neural networks train significantly better on HDR and RAW images represented in display color spaces. This small change to the training strategy can bring a very substantial gain in performance, up to 10-15 dB.
  arXiv  Detail & Related papers  (2023-12-06T17:47:16Z)
• UniGS: Unified Representation for Image Generation and Segmentation [105.08152635402858]
  We use a colormap to represent entity-level masks, addressing the challenge of varying entity numbers. Two novel modules, including the location-aware color palette and progressive dichotomy module, are proposed to support our mask representation.
  arXiv  Detail & Related papers  (2023-12-04T15:59:27Z)
• Unsupervised HDR Image and Video Tone Mapping via Contrastive Learning [19.346284003982035]
  We propose a unified framework (IVTMNet) for unsupervised image and video tone mapping. For video tone mapping, we propose a temporal-feature-replaced (TFR) module to efficiently utilize the temporal correlation. Experimental results demonstrate that our method outperforms state-of-the-art image and video tone mapping methods.
  arXiv  Detail & Related papers  (2023-03-13T17:45:39Z)
• Transform your Smartphone into a DSLR Camera: Learning the ISP in the Wild [159.71025525493354]
  We propose a trainable Image Signal Processing framework that produces DSLR quality images given RAW images captured by a smartphone. To address the color misalignments between training image pairs, we employ a color-conditional ISP network and optimize a novel parametric color mapping between each input RAW and reference DSLR image.
  arXiv  Detail & Related papers  (2022-03-20T20:13:59Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Wavelet-Based Dual-Branch Network for Image Demoireing [148.91145614517015]
  We design a wavelet-based dual-branch network (WDNet) with a spatial attention mechanism for image demoireing. Our network removes moire patterns in the wavelet domain to separate the frequencies of moire patterns from the image content. Experiments demonstrate the effectiveness of our method, and we further show that WDNet generalizes to removing moire artifacts on non-screen images.
  arXiv  Detail & Related papers  (2020-07-14T16:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.