Data-Side Efficiencies for Lightweight Convolutional Neural Networks

• URL: http://arxiv.org/abs/2308.13057v1
• Date: Thu, 24 Aug 2023 19:50:25 GMT
• Title: Data-Side Efficiencies for Lightweight Convolutional Neural Networks
• Authors: Bryan Bo Cao, Lawrence O'Gorman, Michael Coss, Shubham Jain
• Abstract summary: We show how four data attributes - number of classes, object color, image resolution, and object scale affect neural network model size and efficiency. We provide an example, applying the metrics and methods to choose a lightweight model for a robot path planning application.
• Score: 4.5853328688992905
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We examine how the choice of data-side attributes for two important visual tasks of image classification and object detection can aid in the choice or design of lightweight convolutional neural networks. We show by experimentation how four data attributes - number of classes, object color, image resolution, and object scale affect neural network model size and efficiency. Intra- and inter-class similarity metrics, based on metric learning, are defined to guide the evaluation of these attributes toward achieving lightweight models. Evaluations made using these metrics are shown to require 30x less computation than running full inference tests. We provide, as an example, applying the metrics and methods to choose a lightweight model for a robot path planning application and achieve computation reduction of 66% and accuracy gain of 3.5% over the pre-method model.

Related papers

• Low-Resolution Neural Networks [0.552480439325792]
  This study examines the impact of parameter bit precision on model performance compared to standard 32-bit models. Models analyzed include those with fully connected layers, convolutional layers, and transformer blocks. Findings suggest a potential new era for optimized neural network models with reduced memory requirements and improved computational efficiency.
  arXiv  Detail & Related papers  (2025-02-12T21:19:28Z)
• Memory-Efficient Graph Convolutional Networks for Object Classification and Detection with Event Cameras [2.3311605203774395]
  Graph convolutional networks (GCNs) are a promising approach for analyzing event data. In this paper, we consider both factors together in order to achieve satisfying results and relatively low model complexity. Our results show a 450-fold reduction in the number of parameters for the feature extraction module and a 4.5-fold reduction in the size of the data representation.
  arXiv  Detail & Related papers  (2023-07-26T11:44:44Z)
• Siamese Networks for Weakly Supervised Human Activity Recognition [2.398608007786179]
  We present a model with multiple siamese networks that are trained by using only the information about the similarity between pairs of data samples without knowing the explicit labels. The trained model maps the activity data samples into fixed size representation vectors such that the distance between the vectors in the representation space approximates the similarity of the data samples in the input space. We evaluate the model on three datasets to verify its effectiveness in segmentation and recognition of continuous human activity sequences.
  arXiv  Detail & Related papers  (2023-07-18T03:23:34Z)
• Learning Customized Visual Models with Retrieval-Augmented Knowledge [104.05456849611895]
  We propose REACT, a framework to acquire the relevant web knowledge to build customized visual models for target domains. We retrieve the most relevant image-text pairs from the web-scale database as external knowledge, and propose to customize the model by only training new modualized blocks while freezing all the original weights. The effectiveness of REACT is demonstrated via extensive experiments on classification, retrieval, detection and segmentation tasks, including zero, few, and full-shot settings.
  arXiv  Detail & Related papers  (2023-01-17T18:59:06Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)
• Estimating informativeness of samples with Smooth Unique Information [108.25192785062367]
  We measure how much a sample informs the final weights and how much it informs the function computed by the weights. We give efficient approximations of these quantities using a linearized network. We apply these measures to several problems, such as dataset summarization.
  arXiv  Detail & Related papers  (2021-01-17T10:29:29Z)
• Fast Uncertainty Quantification for Deep Object Pose Estimation [91.09217713805337]
  Deep learning-based object pose estimators are often unreliable and overconfident. In this work, we propose a simple, efficient, and plug-and-play UQ method for 6-DoF object pose estimation.
  arXiv  Detail & Related papers  (2020-11-16T06:51:55Z)
• ReMarNet: Conjoint Relation and Margin Learning for Small-Sample Image Classification [49.87503122462432]
  We introduce a novel neural network termed Relation-and-Margin learning Network (ReMarNet) Our method assembles two networks of different backbones so as to learn the features that can perform excellently in both of the aforementioned two classification mechanisms. Experiments on four image datasets demonstrate that our approach is effective in learning discriminative features from a small set of labeled samples.
  arXiv  Detail & Related papers  (2020-06-27T13:50:20Z)
• LiteDenseNet: A Lightweight Network for Hyperspectral Image Classification [2.696926374562295]
  We propose a lightweight network architecture (LiteDenseNet) based on DenseNet for Hyperspectral Image Classification. Inspired by GoogLeNet and PeleeNet, we design a 3D two-way dense layer to capture the local and global features of the input. As convolution is a computationally intensive operation, we introduce group convolution to decrease calculation cost and parameter size further.
  arXiv  Detail & Related papers  (2020-04-17T08:38:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.