A Deep Neural Network Based Approach to Building Budget-Constrained Models for Big Data Analysis

• URL: http://arxiv.org/abs/2302.11707v1
• Date: Thu, 23 Feb 2023 00:00:32 GMT
• Title: A Deep Neural Network Based Approach to Building Budget-Constrained Models for Big Data Analysis
• Authors: Rui Ming, Haiping Xu, Shannon E. Gibbs, Donghui Yan, Ming Shao
• Abstract summary: We introduce an approach to eliminating less important features for big data analysis using Deep Neural Networks (DNNs) We identify the weak links and weak neurons, and remove some input features to bring the model cost within a given budget.
• Score: 11.562071835482223
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning approaches require collection of data on many different input features or variables for accurate model training and prediction. Since data collection on input features could be costly, it is crucial to reduce the cost by selecting a subset of features and developing a budget-constrained model (BCM). In this paper, we introduce an approach to eliminating less important features for big data analysis using Deep Neural Networks (DNNs). Once a DNN model has been developed, we identify the weak links and weak neurons, and remove some input features to bring the model cost within a given budget. The experimental results show our approach is feasible and supports user selection of a suitable BCM within a given budget.

Related papers

• How Much Data are Enough? Investigating Dataset Requirements for Patch-Based Brain MRI Segmentation Tasks [74.21484375019334]
  Training deep neural networks reliably requires access to large-scale datasets. To mitigate both the time and financial costs associated with model development, a clear understanding of the amount of data required to train a satisfactory model is crucial. This paper proposes a strategic framework for estimating the amount of annotated data required to train patch-based segmentation networks.
  arXiv  Detail & Related papers  (2024-04-04T13:55:06Z)
• Scalable Learning of Item Response Theory Models [53.43355949923962]
  Item Response Theory (IRT) models aim to assess latent abilities of $n$ examinees along with latent difficulty characteristics of $m$ test items from categorical data. We leverage the similarity of these models to logistic regression, which can be approximated accurately using small weighted subsets called coresets.
  arXiv  Detail & Related papers  (2024-03-01T17:12:53Z)
• A Systematic Approach to Robustness Modelling for Deep Convolutional Neural Networks [0.294944680995069]
  Recent work raises questions about the ability for even larger models to generalize to data outside of the controlled train and test sets. We provide a method that uses induced failures to model the probability of failure as a function of time. We examine the various trade-offs between cost, robustness, latency, and reliability to find that larger models do not significantly aid in adversarial robustness.
  arXiv  Detail & Related papers  (2024-01-24T19:12:37Z)
• Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective [67.25782152459851]
  We introduce a novel data-model co-design perspective: to promote superior weight sparsity. Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
  arXiv  Detail & Related papers  (2023-12-03T13:50:24Z)
• A Tale of Two Cities: Data and Configuration Variances in Robust Deep Learning [27.498927971861068]
  Deep neural networks (DNNs) are widely used in many industries such as image recognition, supply chain, medical diagnosis, and autonomous driving. Prior work has shown the high accuracy of a DNN model does not imply high robustness because the input data and external environment are constantly changing.
  arXiv  Detail & Related papers  (2022-11-18T03:32:53Z)
• Rethinking Cost-sensitive Classification in Deep Learning via Adversarial Data Augmentation [4.479834103607382]
  Cost-sensitive classification is critical in applications where misclassification errors widely vary in cost. This paper proposes a cost-sensitive adversarial data augmentation framework to make over- parameterized models cost-sensitive. Our method can effectively minimize the overall cost and reduce critical errors, while achieving comparable performance in terms of overall accuracy.
  arXiv  Detail & Related papers  (2022-08-24T19:00:30Z)
• RoMA: Robust Model Adaptation for Offline Model-based Optimization [115.02677045518692]
  We consider the problem of searching an input maximizing a black-box objective function given a static dataset of input-output queries. A popular approach to solving this problem is maintaining a proxy model that approximates the true objective function. Here, the main challenge is how to avoid adversarially optimized inputs during the search.
  arXiv  Detail & Related papers  (2021-10-27T05:37:12Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
  We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
  arXiv  Detail & Related papers  (2020-03-04T12:27:36Z)
• FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization Approach for Deep Neural Networks [4.596221278839825]
  We develop a novel multi-objective optimization algorithm, we call Flexible Multi-Objective Bayesian Optimization (FlexiBO) to address this issue. We evaluate FlexiBO on seven state-of-the-art DNNs for image recognition, natural language processing (NLP), and speech-to-text translation.
  arXiv  Detail & Related papers  (2020-01-18T03:26:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.