Revisiting Global Statistics Aggregation for Improving Image Restoration

• URL: http://arxiv.org/abs/2112.04491v1
• Date: Wed, 8 Dec 2021 12:52:14 GMT
• Title: Revisiting Global Statistics Aggregation for Improving Image Restoration
• Authors: Xiaojie Chu, Liangyu Chen, Chengpeng Chen, Xin Lu
• Abstract summary: Test-time Local Statistics Converter (TLSC) significantly improves image restorer's performance. By extending SE with TLSC to the state-of-the-art models, MPRNet boost by 0.65 dB in PSNR on GoPro dataset, achieves 33.31 dB, exceeds the previous best result 0.6 dB.
• Score: 8.803962179239385
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Global spatial statistics, which are aggregated along entire spatial dimensions, are widely used in top-performance image restorers. For example, mean, variance in Instance Normalization (IN) which is adopted by HINet, and global average pooling (i.e. mean) in Squeeze and Excitation (SE) which is applied to MPRNet. This paper first shows that statistics aggregated on the patches-based/entire-image-based feature in the training/testing phase respectively may distribute very differently and lead to performance degradation in image restorers. It has been widely overlooked by previous works. To solve this issue, we propose a simple approach, Test-time Local Statistics Converter (TLSC), that replaces the region of statistics aggregation operation from global to local, only in the test time. Without retraining or finetuning, our approach significantly improves the image restorer's performance. In particular, by extending SE with TLSC to the state-of-the-art models, MPRNet boost by 0.65 dB in PSNR on GoPro dataset, achieves 33.31 dB, exceeds the previous best result 0.6 dB. In addition, we simply apply TLSC to the high-level vision task, i.e. semantic segmentation, and achieves competitive results. Extensive quantity and quality experiments are conducted to demonstrate TLSC solves the issue with marginal costs while significant gain. The code is available at https://github.com/megvii-research/tlsc.

Related papers

• AIR-HLoc: Adaptive Retrieved Images Selection for Efficient Visual Localisation [8.789742514363777]
  State-of-the-art hierarchical localisation pipelines (HLoc) employ image retrieval (IR) to establish 2D-3D correspondences. This paper investigates the relationship between global and local descriptors. We propose an adaptive strategy that adjusts $k$ based on the similarity between the query's global descriptor and those in the database.
  arXiv  Detail & Related papers  (2024-03-27T06:17:21Z)
• Global Features are All You Need for Image Retrieval and Reranking [2.6198864241281434]
  SuperGlobal is a novel approach that exclusively employs global features for both stages, improving efficiency without sacrificing accuracy. Our experiments demonstrate substantial improvements compared to the state of the art in standard benchmarks. Our two-stage system surpasses the current single-stage state-of-the-art by 16.3%, offering a scalable, accurate alternative for high-performing image retrieval systems.
  arXiv  Detail & Related papers  (2023-08-14T06:13:27Z)
• Patch-aware Batch Normalization for Improving Cross-domain Robustness [55.06956781674986]
  Cross-domain tasks present a challenge in which the model's performance will degrade when the training set and the test set follow different distributions. We propose a novel method called patch-aware batch normalization (PBN) By exploiting the differences between local patches of an image, our proposed PBN can effectively enhance the robustness of the model's parameters.
  arXiv  Detail & Related papers  (2023-04-06T03:25:42Z)
• Inverse Image Frequency for Long-tailed Image Recognition [59.40098825416675]
  We propose a novel de-biasing method named Inverse Image Frequency (IIF) IIF is a multiplicative margin adjustment transformation of the logits in the classification layer of a convolutional neural network. Our experiments show that IIF surpasses the state of the art on many long-tailed benchmarks.
  arXiv  Detail & Related papers  (2022-09-11T13:31:43Z)
• Self-Supervised Pre-Training for Transformer-Based Person Re-Identification [54.55281692768765]
  Transformer-based supervised pre-training achieves great performance in person re-identification (ReID) Due to the domain gap between ImageNet and ReID datasets, it usually needs a larger pre-training dataset to boost the performance. This work aims to mitigate the gap between the pre-training and ReID datasets from the perspective of data and model structure.
  arXiv  Detail & Related papers  (2021-11-23T18:59:08Z)
• Test-time Batch Statistics Calibration for Covariate Shift [66.7044675981449]
  We propose to adapt the deep models to the novel environment during inference. We present a general formulation $alpha$-BN to calibrate the batch statistics. We also present a novel loss function to form a unified test time adaptation framework Core.
  arXiv  Detail & Related papers  (2021-10-06T08:45:03Z)
• Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classification [97.81205777897043]
  Recent work has shown that convolutional neural network classifiers overly rely on texture at the expense of shape cues. We make a similar but different distinction between shape and local image cues, on the one hand, and global image statistics, on the other. Our method, called Permuted Adaptive Instance Normalization (pAdaIN), reduces the representation of global statistics in the hidden layers of image classifiers.
  arXiv  Detail & Related papers  (2020-10-09T16:38:38Z)
• Tent: Fully Test-time Adaptation by Entropy Minimization [77.85911673550851]
  A model must adapt itself to generalize to new and different data during testing. In this setting of fully test-time adaptation the model has only the test data and its own parameters. We propose to adapt by test entropy minimization (tent): we optimize the model for confidence as measured by the entropy of its predictions.
  arXiv  Detail & Related papers  (2020-06-18T17:55:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.