Characterizing and Taming Resolution in Convolutional Neural Networks

• URL: http://arxiv.org/abs/2110.14819v1
• Date: Thu, 28 Oct 2021 00:08:23 GMT
• Title: Characterizing and Taming Resolution in Convolutional Neural Networks
• Authors: Eddie Yan, Liang Luo, Luis Ceze
• Abstract summary: Image resolution has a significant effect on the accuracy and computational, storage, and bandwidth costs of computer vision model inference. We study the accuracy and efficiency tradeoff via systematic and automated tuning of image resolution, image quality and convolutional neural network operators. We propose a dynamic resolution mechanism that removes the need to statically choose a resolution ahead of time.
• Score: 4.412616624011115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image resolution has a significant effect on the accuracy and computational, storage, and bandwidth costs of computer vision model inference. These costs are exacerbated when scaling out models to large inference serving systems and make image resolution an attractive target for optimization. However, the choice of resolution inherently introduces additional tightly coupled choices, such as image crop size, image detail, and compute kernel implementation that impact computational, storage, and bandwidth costs. Further complicating this setting, the optimal choices from the perspective of these metrics are highly dependent on the dataset and problem scenario. We characterize this tradeoff space, quantitatively studying the accuracy and efficiency tradeoff via systematic and automated tuning of image resolution, image quality and convolutional neural network operators. With the insights from this study, we propose a dynamic resolution mechanism that removes the need to statically choose a resolution ahead of time.

Related papers

• Elastic-DETR: Making Image Resolution Learnable with Content-Specific Network Prediction [0.612477318852572]
  We introduce a novel strategy for learnable resolution, called Elastic-DETR, enabling elastic utilization of multiple image resolutions. Our network provides an adaptive scale factor based on the content of the image with a compact scale prediction module. By leveraging the resolution's flexibility, we can demonstrate various models that exhibit varying trade-offs between accuracy and computational complexity.
  arXiv  Detail & Related papers  (2024-12-09T09:46:21Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
  Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration. We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
  arXiv  Detail & Related papers  (2024-05-23T09:13:36Z)
• VmambaIR: Visual State Space Model for Image Restoration [36.11385876754612]
  We propose VmambaIR, which introduces State Space Models (SSMs) with linear complexity into comprehensive image restoration tasks. VmambaIR achieves state-of-the-art (SOTA) performance with much fewer computational resources and parameters.
  arXiv  Detail & Related papers  (2024-03-18T02:38:55Z)
• DyRA: Portable Dynamic Resolution Adjustment Network for Existing Detectors [0.669087470775851]
  This paper introduces DyRA, a dynamic resolution adjustment network providing an image-specific scale factor for existing detectors. Loss function is devised to minimize the accuracy drop across contrasting objectives of different-sized objects for scaling.
  arXiv  Detail & Related papers  (2023-11-28T07:52:41Z)
• Semantically Accurate Super-Resolution Generative Adversarial Networks [2.0454959820861727]
  We propose a novel architecture and domain-specific feature loss to increase the performance of semantic segmentation. We show the proposed approach improves perceived image quality as well as quantitative segmentation accuracy across all prediction classes. This work demonstrates that jointly considering image-based and task-specific losses can improve the performance of both, and advances the state-of-the-art in semantic-aware super-resolution of aerial imagery.
  arXiv  Detail & Related papers  (2022-05-17T23:05:27Z)
• High Quality Segmentation for Ultra High-resolution Images [72.97958314291648]
  We propose the Continuous Refinement Model for the ultra high-resolution segmentation refinement task. Our proposed method is fast and effective on image segmentation refinement.
  arXiv  Detail & Related papers  (2021-11-29T11:53:06Z)
• Dynamic Resolution Network [40.64164953983429]
  The redundancy on the input resolution of modern CNNs has not been fully investigated. We propose a novel dynamic-resolution network (DRNet) in which the resolution is determined dynamically based on each input sample. DRNet achieves similar performance with an about 34% reduction, while gains 1.4% accuracy increase with 10% reduction compared to the original ResNet-50 on ImageNet.
  arXiv  Detail & Related papers  (2021-06-05T13:48:33Z)
• Enhancing sensor resolution improves CNN accuracy given the same number of parameters or FLOPS [53.10151901863263]
  We show that it is almost always possible to modify a network such that it achieves higher accuracy at a higher input resolution while having the same number of parameters or/and FLOPS. Preliminary empirical investigation over MNIST, Fashion MNIST, and CIFAR10 datasets demonstrates the efficiency of the proposed approach.
  arXiv  Detail & Related papers  (2021-03-09T06:47:01Z)
• Invertible Image Rescaling [118.2653765756915]
  We develop an Invertible Rescaling Net (IRN) to produce visually-pleasing low-resolution images. We capture the distribution of the lost information using a latent variable following a specified distribution in the downscaling process.
  arXiv  Detail & Related papers  (2020-05-12T09:55:53Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.