Global Adaptive Filtering Layer for Computer Vision

• URL: http://arxiv.org/abs/2010.01177v4
• Date: Wed, 4 Aug 2021 15:52:46 GMT
• Title: Global Adaptive Filtering Layer for Computer Vision
• Authors: Viktor Shipitsin, Iaroslav Bespalov, Dmitry V. Dylov
• Abstract summary: We devise a universal adaptive neural layer to "learn" optimal frequency filter for each image together with the weights of the base neural network that performs some computer vision task. The proposed approach takes the source image in the spatial domain, automatically selects the best frequencies from the frequency domain, and transmits the inverse-transform image to the main neural network. We observe that the light networks gain a noticeable boost in the performance metrics; whereas, the training of the heavy ones converges faster when our adaptive layer is allowed to "learn" alongside the main architecture.
• Score: 2.2758845733923687
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We devise a universal adaptive neural layer to "learn" optimal frequency filter for each image together with the weights of the base neural network that performs some computer vision task. The proposed approach takes the source image in the spatial domain, automatically selects the best frequencies from the frequency domain, and transmits the inverse-transform image to the main neural network. Remarkably, such a simple add-on layer dramatically improves the performance of the main network regardless of its design. We observe that the light networks gain a noticeable boost in the performance metrics; whereas, the training of the heavy ones converges faster when our adaptive layer is allowed to "learn" alongside the main architecture. We validate the idea in four classical computer vision tasks: classification, segmentation, denoising, and erasing, considering popular natural and medical data benchmarks.

Related papers

• On the universality of neural encodings in CNNs [5.064404027153094]
  We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. They explain, at a more fundamental level, the success of transfer learning.
  arXiv  Detail & Related papers  (2024-09-28T21:30:25Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• A Scalable Walsh-Hadamard Regularizer to Overcome the Low-degree Spectral Bias of Neural Networks [79.28094304325116]
  Despite the capacity of neural nets to learn arbitrary functions, models trained through gradient descent often exhibit a bias towards simpler'' functions. We show how this spectral bias towards low-degree frequencies can in fact hurt the neural network's generalization on real-world datasets. We propose a new scalable functional regularization scheme that aids the neural network to learn higher degree frequencies.
  arXiv  Detail & Related papers  (2023-05-16T20:06:01Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Increasing the Accuracy of a Neural Network Using Frequency Selective Mesh-to-Grid Resampling [4.211128681972148]
  We propose the use of keypoint frequency selective mesh-to-grid resampling (FSMR) for the processing of input data for neural networks. We show that depending on the network architecture and classification task the application of FSMR during training aids learning process. The classification accuracy can be increased by up to 4.31 percentage points for ResNet50 and the Oxflower17 dataset.
  arXiv  Detail & Related papers  (2022-09-28T21:34:47Z)
• Frequency Dropout: Feature-Level Regularization via Randomized Filtering [24.53978165468098]
  Deep convolutional neural networks are susceptible to picking up spurious correlations from the training signal. We propose a training strategy, Frequency Dropout, to prevent convolutional neural networks from learning frequency-specific imaging features. Our results suggest that the proposed approach does not only improve predictive accuracy but also improves robustness against domain shift.
  arXiv  Detail & Related papers  (2022-09-20T16:42:21Z)
• Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition [77.95361323613147]
  Current state-of-the-art visual recognition systems rely on pretraining a neural network on a large-scale dataset and finetuning the network weights on a smaller dataset. In this work, we prove that dynamically adapting network architectures tailored for each domain task along with weight finetuning benefits in both efficiency and effectiveness. Our method can be easily generalized to an unsupervised paradigm by replacing supernet training with self-supervised learning in the source domain tasks and performing linear evaluation in the downstream tasks.
  arXiv  Detail & Related papers  (2021-03-31T08:15:17Z)
• Frequency learning for image classification [1.9336815376402716]
  This paper presents a new approach for exploring the Fourier transform of the input images, which is composed of trainable frequency filters. We propose a slicing procedure to allow the network to learn both global and local features from the frequency-domain representations of the image blocks.
  arXiv  Detail & Related papers  (2020-06-28T00:32:47Z)
• Large-scale spatiotemporal photonic reservoir computer for image classification [0.8701566919381222]
  We propose a scalable photonic architecture for implementation of feedforward and recurrent neural networks to perform the classification of handwritten digits. Our experiment exploits off-the-shelf optical and electronic components to currently achieve a network size of 16,384 nodes.
  arXiv  Detail & Related papers  (2020-04-06T10:22:31Z)
• Curriculum By Smoothing [52.08553521577014]
  Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. We propose an elegant curriculum based scheme that smoothes the feature embedding of a CNN using anti-aliasing or low-pass filters. As the amount of information in the feature maps increases during training, the network is able to progressively learn better representations of the data.
  arXiv  Detail & Related papers  (2020-03-03T07:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.