Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling

• URL: http://arxiv.org/abs/2307.09804v1
• Date: Wed, 19 Jul 2023 07:47:23 GMT
• Title: Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling
• Authors: Julia Grabinski, Janis Keuper and Margret Keuper
• Abstract summary: Convolutional neural networks encode images through a sequence of convolutions, normalizations and non-linearities as well as downsampling operations. Previous work showed that even slight mistakes during sampling, leading to aliasing, can be directly attributed to the networks' lack in robustness. We propose aliasing and spectral artifact-free pooling, short ASAP.
• Score: 11.72025865314187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional neural networks encode images through a sequence of convolutions, normalizations and non-linearities as well as downsampling operations into potentially strong semantic embeddings. Yet, previous work showed that even slight mistakes during sampling, leading to aliasing, can be directly attributed to the networks' lack in robustness. To address such issues and facilitate simpler and faster adversarial training, [12] recently proposed FLC pooling, a method for provably alias-free downsampling - in theory. In this work, we conduct a further analysis through the lens of signal processing and find that such current pooling methods, which address aliasing in the frequency domain, are still prone to spectral leakage artifacts. Hence, we propose aliasing and spectral artifact-free pooling, short ASAP. While only introducing a few modifications to FLC pooling, networks using ASAP as downsampling method exhibit higher native robustness against common corruptions, a property that FLC pooling was missing. ASAP also increases native robustness against adversarial attacks on high and low resolution data while maintaining similar clean accuracy or even outperforming the baseline.

Related papers

• Freq-Mip-AA : Frequency Mip Representation for Anti-Aliasing Neural Radiance Fields [3.796287987989994]
  Mip-NeRF proposed using frustums to render a pixel and suggested integrated positional encoding (IPE) While effective, this approach requires long training times due to its reliance on volumetric architecture. We propose a novel anti-aliasing technique that utilizes grid-based representations, usually showing significantly faster training time.
  arXiv  Detail & Related papers  (2024-06-19T06:33:56Z)
• Arbitrary-Scale Point Cloud Upsampling by Voxel-Based Network with Latent Geometric-Consistent Learning [52.825441454264585]
  We propose an arbitrary-scale Point cloud Upsampling framework using Voxel-based Network (textbfPU-VoxelNet) Thanks to the completeness and regularity inherited from the voxel representation, voxel-based networks are capable of providing predefined grid space to approximate 3D surface. A density-guided grid resampling method is developed to generate high-fidelity points while effectively avoiding sampling outliers.
  arXiv  Detail & Related papers  (2024-03-08T07:31:14Z)
• Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction [105.46091601932524]
  We introduce Fine Structured-Aware Sampling (FSS) to train pixel-aligned implicit models for single-view human reconstruction. FSS proactively adapts to the thickness and complexity of surfaces. It also proposes a mesh thickness loss signal for pixel-aligned implicit models.
  arXiv  Detail & Related papers  (2024-02-29T14:26:46Z)
• Aliasing is a Driver of Adversarial Attacks [35.262520934751]
  We investigate the hypothesis that the existence of adversarial perturbations is due in part to aliasing in neural networks. Our ultimate goal is to increase robustness against adversarial attacks using explainable, non-trained, structural changes only. Our experimental results show a solid link between anti-aliasing and adversarial attacks.
  arXiv  Detail & Related papers  (2022-12-22T14:52:44Z)
• Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging [142.11622043078867]
  We propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST) for HSI reconstruction.
  arXiv  Detail & Related papers  (2022-05-20T11:37:44Z)
• FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting [12.062691258844628]
  This paper introduces an aliasing free down-sampling operation which can easily be plugged into any CNN architecture. Our experiments show, that in combination with simple and fast FGSM adversarial training, our hyper- parameter free operator significantly improves model robustness.
  arXiv  Detail & Related papers  (2022-04-01T14:51:28Z)
• Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction [138.04956118993934]
  We propose a novel Transformer-based method, coarse-to-fine sparse Transformer (CST) CST embedding HSI sparsity into deep learning for HSI reconstruction. In particular, CST uses our proposed spectra-aware screening mechanism (SASM) for coarse patch selecting. Then the selected patches are fed into our customized spectra-aggregation hashing multi-head self-attention (SAH-MSA) for fine pixel clustering and self-similarity capturing.
  arXiv  Detail & Related papers  (2022-03-09T16:17:47Z)
• Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Function [7.888131635057012]
  Deep convolutional networks are vulnerable to image translation or shift. The textbook solution is low-pass filtering before down-sampling. We show that Depth Adaptive Blurring is more effective, as opposed to monotonic blurring.
  arXiv  Detail & Related papers  (2021-10-03T01:00:52Z)
• Normalized Convolution Upsampling for Refined Optical Flow Estimation [23.652615797842085]
  Normalized Convolution UPsampler (NCUP) is an efficient joint upsampling approach to produce the full-resolution flow during the training of optical flow CNNs. Our proposed approach formulates the upsampling task as a sparse problem and employs the normalized convolutional neural networks to solve it. We achieve state-of-the-art results on Sintel benchmark with 6% error reduction, and on-par on the KITTI dataset, while having 7.5% fewer parameters.
  arXiv  Detail & Related papers  (2021-02-13T18:34:03Z)
• Salvage Reusable Samples from Noisy Data for Robust Learning [70.48919625304]
  We propose a reusable sample selection and correction approach, termed as CRSSC, for coping with label noise in training deep FG models with web images. Our key idea is to additionally identify and correct reusable samples, and then leverage them together with clean examples to update the networks.
  arXiv  Detail & Related papers  (2020-08-06T02:07:21Z)
• Cascaded Regression Tracking: Towards Online Hard Distractor Discrimination [202.2562153608092]
  We propose a cascaded regression tracker with two sequential stages. In the first stage, we filter out abundant easily-identified negative candidates. In the second stage, a discrete sampling based ridge regression is designed to double-check the remaining ambiguous hard samples.
  arXiv  Detail & Related papers  (2020-06-18T07:48:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.