Adaptive Network Combination for Single-Image Reflection Removal: A Domain Generalization Perspective

• URL: http://arxiv.org/abs/2204.01505v1
• Date: Mon, 4 Apr 2022 14:06:11 GMT
• Title: Adaptive Network Combination for Single-Image Reflection Removal: A Domain Generalization Perspective
• Authors: Ming Liu, Jianan Pan, Zifei Yan, Wangmeng Zuo, Lei Zhang
• Abstract summary: In this paper, we tackle issues by learning SIRR models from a domain perspective. For each source set, a specific SIRR model is trained to serve as a domain expert of relevant reflection types. For images from one source set, we train RTAW to only predict expert-wise weights of other domain experts for improving generalization ability. Experiments show the appealing performance gain of our AdaNEC on different state-of-the-art SIRR networks.
• Score: 68.37624784559728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, multiple synthetic and real-world datasets have been built to facilitate the training of deep single image reflection removal (SIRR) models. Meanwhile, diverse testing sets are also provided with different types of reflection and scenes. However, the non-negligible domain gaps between training and testing sets make it difficult to learn deep models generalizing well to testing images. The diversity of reflections and scenes further makes it a mission impossible to learn a single model being effective to all testing sets and real-world reflections. In this paper, we tackle these issues by learning SIRR models from a domain generalization perspective. Particularly, for each source set, a specific SIRR model is trained to serve as a domain expert of relevant reflection types. For a given reflection-contaminated image, we present a reflection type-aware weighting (RTAW) module to predict expert-wise weights. RTAW can then be incorporated with adaptive network combination (AdaNEC) for handling different reflection types and scenes, i.e., generalizing to unknown domains. Two representative AdaNEC methods, i.e., output fusion (OF) and network interpolation (NI), are provided by considering both adaptation levels and efficiency. For images from one source set, we train RTAW to only predict expert-wise weights of other domain experts for improving generalization ability, while the weights of all experts are predicted and employed during testing. An in-domain expert (IDE) loss is presented for training RTAW. Extensive experiments show the appealing performance gain of our AdaNEC on different state-of-the-art SIRR networks. Source code and pre-trained models will available at https://github.com/csmliu/AdaNEC.

Related papers

• CoDEx: Combining Domain Expertise for Spatial Generalization in Satellite Image Analysis [20.904517823908783]
  We propose a novel domain-generalization framework for satellite images. We train one expert model per training domain, while learning experts' similarity and encouraging similar experts to be consistent. A model selection module then identifies the most suitable experts for a given test sample and aggregates their predictions.
  arXiv  Detail & Related papers  (2025-04-28T12:33:39Z)
• Dereflection Any Image with Diffusion Priors and Diversified Data [86.15504914121226]
  We propose a comprehensive solution with an efficient data preparation pipeline and a generalizable model for robust reflection removal. First, we introduce a dataset named Diverse Reflection Removal (DRR) created by randomly rotating reflective mediums in target scenes. Second, we propose a diffusion-based framework with one-step diffusion for deterministic outputs and fast inference.
  arXiv  Detail & Related papers  (2025-03-21T17:48:14Z)
• A Lightweight Deep Exclusion Unfolding Network for Single Image Reflection Removal [68.0573194557999]
  Single Image Reflection Removal (SIRR) is a canonical blind source separation problem. We propose a novel Deep Exclusion unfolding Network (DExNet) for SIRR. DExNet is constructed by unfolding and parameterizing a simple iterative Sparse and Auxiliary Feature Update (i-SAFU) algorithm.
  arXiv  Detail & Related papers  (2025-03-03T07:54:27Z)
• PromptMID: Modal Invariant Descriptors Based on Diffusion and Vision Foundation Models for Optical-SAR Image Matching [15.840638449527399]
  We propose PromptMID, a novel approach that constructs modality-invariant descriptors using text prompts. PromptMID extracts multi-scale modality-invariant features by leveraging pre-trained diffusion models and visual foundation models. Experiments on optical-SAR image datasets from four diverse regions demonstrate that PromptMID outperforms state-of-the-art matching methods.
  arXiv  Detail & Related papers  (2025-02-25T11:19:26Z)
• Neural Experts: Mixture of Experts for Implicit Neural Representations [41.395193251292895]
  Implicit neural representations (INRs) have proven effective in various tasks including image, shape, audio, and video reconstruction. We propose a mixture of experts (MoE) implicit neural representation approach that enables learning local piece-wise continuous functions. We show that incorporating a mixture of experts architecture into existing INR formulations provides a boost in speed, accuracy, and memory requirements.
  arXiv  Detail & Related papers  (2024-10-29T01:11:25Z)
• Context Enhancement with Reconstruction as Sequence for Unified Unsupervised Anomaly Detection [68.74469657656822]
  Unsupervised anomaly detection (AD) aims to train robust detection models using only normal samples. Recent research focuses on a unified unsupervised AD setting in which only one model is trained for all classes. We introduce a novel Reconstruction as Sequence (RAS) method, which enhances the contextual correspondence during feature reconstruction.
  arXiv  Detail & Related papers  (2024-09-10T07:37:58Z)
• Reflection Invariance Learning for Few-shot Semantic Segmentation [53.20466630330429]
  Few-shot semantic segmentation (FSS) aims to segment objects of unseen classes in query images with only a few annotated support images. This paper proposes a fresh few-shot segmentation framework to mine the reflection invariance in a multi-view matching manner. Experiments on both PASCAL-$5textiti$ and COCO-$20textiti$ datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-01T15:14:58Z)
• Adaptive adversarial training method for improving multi-scale GAN based on generalization bound theory [14.562893624131531]
  This paper pioneered the introduction of PAC-Bayes generalized bound theory into the training analysis of specific models. We proposed an adaptive training method which can greatly improve the image manipulation ability of multi-scale GANs. In particular, for the image super-resolution restoration task, the multi-scale GAN model trained by the proposed method achieves a 100% reduction in natural image quality evaluator (NIQE) and a 60% reduction in root mean squared error (RMSE)
  arXiv  Detail & Related papers  (2022-11-30T07:11:56Z)
• Diversified Dynamic Routing for Vision Tasks [36.199659460868496]
  We propose a novel architecture where each layer is composed of a set of experts. In our method, the model is explicitly trained to solve the challenge of finding relevant partitioning of the data. We conduct several experiments on semantic segmentation on Cityscapes and object detection and instance segmentation on MS-COCO.
  arXiv  Detail & Related papers  (2022-09-26T23:27:51Z)
• Two-Stage Single Image Reflection Removal with Reflection-Aware Guidance [78.34235841168031]
  We present a novel two-stage network with reflection-aware guidance (RAGNet) for single image reflection removal (SIRR) RAG can be used (i) to mitigate the effect of reflection from the observation, and (ii) to generate mask in partial convolution for mitigating the effect of deviating from linear combination hypothesis. Experiments on five commonly used datasets demonstrate the quantitative and qualitative superiority of our RAGNet in comparison to the state-of-the-art SIRR methods.
  arXiv  Detail & Related papers  (2020-12-02T03:14:57Z)
• SIR: Self-supervised Image Rectification via Seeing the Same Scene from Multiple Different Lenses [82.56853587380168]
  We propose a novel self-supervised image rectification (SIR) method based on an important insight that the rectified results of distorted images of the same scene from different lens should be the same. We leverage a differentiable warping module to generate the rectified images and re-distorted images from the distortion parameters. Our method achieves comparable or even better performance than the supervised baseline method and representative state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-30T08:23:25Z)
• Unsupervised Single-Image Reflection Separation Using Perceptual Deep Image Priors [6.333390830515411]
  We propose a novel unsupervised framework for single-image reflection separation. We optimize the parameters of two cross-coupled deep convolutional networks on a target image to generate two exclusive background and reflection layers. Our results show that our method significantly outperforms the closest unsupervised method in the literature for removing reflections from single images.
  arXiv  Detail & Related papers  (2020-09-01T21:08:30Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.