Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining

• URL: http://arxiv.org/abs/2502.12600v1
• Date: Tue, 18 Feb 2025 07:23:22 GMT
• Title: Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining
• Authors: Jinfan Hu, Zhiyuan You, Jinjin Gu, Kaiwen Zhu, Tianfan Xue, Chao Dong,
• Abstract summary: Generalization remains a significant challenge for low-level vision models. Image deraining is selected as a case study due to its well-defined and easily decoupled structure. Our findings show that guiding networks to focus on learning the underlying image content, rather than the degradation patterns, is key to improving generalization.
• Score: 26.95998539949823
• License:
• Abstract: Generalization remains a significant challenge for low-level vision models, which often struggle with unseen degradations in real-world scenarios despite their success in controlled benchmarks. In this paper, we revisit the generalization problem in low-level vision models. Image deraining is selected as a case study due to its well-defined and easily decoupled structure, allowing for more effective observation and analysis. Through comprehensive experiments, we reveal that the generalization issue is not primarily due to limited network capacity but rather the failure of existing training strategies, which leads networks to overfit specific degradation patterns. Our findings show that guiding networks to focus on learning the underlying image content, rather than the degradation patterns, is key to improving generalization. We demonstrate that balancing the complexity of background images and degradations in the training data helps networks better fit the image distribution. Furthermore, incorporating content priors from pre-trained generative models significantly enhances generalization. Experiments on both image deraining and image denoising validate the proposed strategies. We believe the insights and solutions will inspire further research and improve the generalization of low-level vision models.

Related papers

• FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restoration [66.61201445650323]
  Existing methods suffer from a generalization bottleneck in real-world scenarios. We contribute a million-scale dataset with two notable advantages over existing training data. We propose a robust model, FoundIR, to better address a broader range of restoration tasks in real-world scenarios.
  arXiv  Detail & Related papers  (2024-12-02T12:08:40Z)
• A Preliminary Exploration Towards General Image Restoration [48.02907312223344]
  We present a new problem called general image restoration (GIR) which aims to address these challenges within a unified model. GIR covers most individual image restoration tasks (eg, image denoising, deblurring, deraining and super-resolution) and their combinations for general purposes. We conduct a comprehensive evaluation of existing approaches for tackling the GIR challenge, illuminating their strengths and pragmatic challenges.
  arXiv  Detail & Related papers  (2024-08-27T15:31:45Z)
• Navigating Beyond Dropout: An Intriguing Solution Towards Generalizable Image Super Resolution [46.31021254956368]
  We argue that Dropout simultaneously introduces undesirable side-effect that compromises model's capacity to faithfully reconstruct fine details. We present another easy yet effective training strategy that enhances the generalization ability of the model by simply modulating its first and second-order features statistics. Experimental results have shown that our method could serve as a model-agnostic regularization and outperforms Dropout on seven benchmark datasets.
  arXiv  Detail & Related papers  (2024-02-29T07:44:31Z)
• Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models [55.919653720979824]
  This paper focuses on the concept of weak-to-strong generalization, which involves using a weaker model to supervise a stronger one. We introduce a novel and adaptively adjustable loss function for weak-to-strong supervision. Our approach not only exceeds the performance benchmarks set by strong-to-strong generalization but also surpasses the outcomes of fine-tuning strong models with whole datasets.
  arXiv  Detail & Related papers  (2024-02-06T06:30:34Z)
• Networks are Slacking Off: Understanding Generalization Problem in Image Deraining [47.47762916146673]
  Deep deraining networks consistently encounter substantial generalization issues when deployed in real-world applications. Our research offers a valuable perspective and methodology for better understanding the generalization problem in low-level vision tasks.
  arXiv  Detail & Related papers  (2023-05-24T13:27:11Z)
• When Neural Networks Fail to Generalize? A Model Sensitivity Perspective [82.36758565781153]
  Domain generalization (DG) aims to train a model to perform well in unseen domains under different distributions. This paper considers a more realistic yet more challenging scenario, namely Single Domain Generalization (Single-DG) We empirically ascertain a property of a model that correlates strongly with its generalization that we coin as "model sensitivity" We propose a novel strategy of Spectral Adversarial Data Augmentation (SADA) to generate augmented images targeted at the highly sensitive frequencies.
  arXiv  Detail & Related papers  (2022-12-01T20:15:15Z)
• Improving Generalization in Federated Learning by Seeking Flat Minima [23.937135834522145]
  Models trained in federated settings often suffer from degraded performances and fail at generalizing. In this work, we investigate such behavior through the lens of geometry of the loss and Hessian eigenspectrum. Motivated by prior studies connecting the sharpness of the loss surface and the generalization gap, we show that i) training clients locally with Sharpness-Aware Minimization (SAM) or its adaptive version (ASAM) on the server-side can substantially improve generalization.
  arXiv  Detail & Related papers  (2022-03-22T16:01:04Z)
• Benchmarking the Robustness of Instance Segmentation Models [3.1287804585804073]
  This paper presents a comprehensive evaluation of instance segmentation models with respect to real-world image corruptions and out-of-domain image collections. The out-of-domain image evaluation shows the generalization capability of models, an essential aspect of real-world applications. Specifically, this benchmark study includes state-of-the-art network architectures, network backbones, normalization layers, models trained starting from scratch or ImageNet pretrained networks.
  arXiv  Detail & Related papers  (2021-09-02T17:50:07Z)
• Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network [92.01145655155374]
  We present an unsupervised image enhancement generative network (UEGAN) It learns the corresponding image-to-image mapping from a set of images with desired characteristics in an unsupervised manner. Results show that the proposed model effectively improves the aesthetic quality of images.
  arXiv  Detail & Related papers  (2020-12-30T03:22:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.