Impact of Privacy Parameters on Deep Learning Models for Image Classification

• URL: http://arxiv.org/abs/2412.06689v1
• Date: Mon, 09 Dec 2024 17:31:55 GMT
• Title: Impact of Privacy Parameters on Deep Learning Models for Image Classification
• Authors: Basanta Chaulagain,
• Abstract summary: We develop differentially private deep learning models for image classification on CIFAR-10 datasets citecifar10.<n>We analyze the impact of various privacy parameters on model accuracy.<n>Our best performing model to date is EfficientNet with test accuracy of $59.63%$ with the following parameters.
• Score: 0.5439020425819
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The project aims to develop differentially private deep learning models for image classification on CIFAR-10 datasets \cite{cifar10} and analyze the impact of various privacy parameters on model accuracy. We have implemented five different deep learning models, namely ConvNet, ResNet18, EfficientNet, ViT, and DenseNet121 and three supervised classifiers namely K-Nearest Neighbors, Naive Bayes Classifier and Support Vector Machine. We evaluated the performance of these models under varying settings. Our best performing model to date is EfficientNet with test accuracy of $59.63\%$ with the following parameters (Adam optimizer, batch size 256, epoch size 100, epsilon value 5.0, learning rate $1e-3$, clipping threshold 1.0, and noise multiplier 0.912).

Related papers

• Impact of Hyperparameter Optimization on the Accuracy of Lightweight Deep Learning Models for Real-Time Image Classification [0.0]
  This work analyzes the influence of hyper parameter adjustment on the accuracy and convergence behavior of seven efficient deep learning architectures.<n>All models are trained on the ImageNet-1K dataset under consistent training settings.<n>Results demonstrate that cosine learning rate decay and adjustable batch size may greatly boost both accuracy and convergence speed.
  arXiv  Detail & Related papers  (2025-07-31T07:47:30Z)
• Comparative Analysis of Lightweight Deep Learning Models for Memory-Constrained Devices [0.0]
  Five state-of-the-art architectures are benchmarked across three diverse datasets: CIFAR-10, CIFAR-100, and Tiny ImageNet.<n>The models are assessed using four key performance metrics: classification accuracy, inference time, floating-point operations (FLOPs), and model size.
  arXiv  Detail & Related papers  (2025-05-06T08:36:01Z)
• Instruction-Following Pruning for Large Language Models [58.329978053711024]
  We move beyond the traditional static pruning approach of determining a fixed pruning mask for a model. In our method, the pruning mask is input-dependent and adapts dynamically based on the information described in a user instruction. Our approach, termed "instruction-following pruning", introduces a sparse mask predictor that takes the user instruction as input and dynamically selects the most relevant model parameters for the given task.
  arXiv  Detail & Related papers  (2025-01-03T20:19:14Z)
• Self-Supervised Learning in Deep Networks: A Pathway to Robust Few-Shot Classification [0.0]
  We first pre-train the model with self-supervision to enable it to learn common feature expressions on a large amount of unlabeled data. Then fine-tune it on the few-shot dataset Mini-ImageNet to improve the model's accuracy and generalization ability under limited data.
  arXiv  Detail & Related papers  (2024-11-19T01:01:56Z)
• PaCKD: Pattern-Clustered Knowledge Distillation for Compressing Memory Access Prediction Models [2.404163279345609]
  PaCKD is a pattern-Clustered Knowledge Distillation approach to compress MAP models. PaCKD yields an 8.70% higher result compared to student models trained with standard knowledge distillation and an 8.88% higher result compared to student models trained without any form of knowledge distillation.
  arXiv  Detail & Related papers  (2024-02-21T00:24:34Z)
• DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classification [19.40810553327253]
  This paper proposes DiffCLIP, a new pre-training framework that incorporates stable diffusion with ControlNet to minimize the domain gap in the visual branch. Experiments on the ModelNet10, ModelNet40, and ScanObjectNN datasets show that DiffCLIP has strong abilities for 3D understanding.
  arXiv  Detail & Related papers  (2023-05-25T11:55:38Z)
• Equivariant Differentially Private Deep Learning: Why DP-SGD Needs Sparser Models [7.49320945341034]
  We show that small and efficient architecture design can outperform current state-of-the-art models with substantially lower computational requirements. Our results are a step towards efficient model architectures that make optimal use of their parameters.
  arXiv  Detail & Related papers  (2023-01-30T17:43:47Z)
• ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders [104.05133094625137]
  We propose a fully convolutional masked autoencoder framework and a new Global Response Normalization layer. This co-design of self-supervised learning techniques and architectural improvement results in a new model family called ConvNeXt V2, which significantly improves the performance of pure ConvNets.
  arXiv  Detail & Related papers  (2023-01-02T18:59:31Z)
• Device-friendly Guava fruit and leaf disease detection using deep learning [0.0]
  This work presents a deep learning-based plant disease diagnostic system using images of fruits and leaves. Five state-of-the-art convolutional neural networks (CNN) have been employed for implementing the system.
  arXiv  Detail & Related papers  (2022-09-26T10:19:57Z)
• SmoothNets: Optimizing CNN architecture design for differentially private deep learning [69.10072367807095]
  DPSGD requires clipping and noising of per-sample gradients. This introduces a reduction in model utility compared to non-private training. We distilled a new model architecture termed SmoothNet, which is characterised by increased robustness to the challenges of DP-SGD training.
  arXiv  Detail & Related papers  (2022-05-09T07:51:54Z)
• Core Risk Minimization using Salient ImageNet [53.616101711801484]
  We introduce the Salient Imagenet dataset with more than 1 million soft masks localizing core and spurious features for all 1000 Imagenet classes. Using this dataset, we first evaluate the reliance of several Imagenet pretrained models (42 total) on spurious features. Next, we introduce a new learning paradigm called Core Risk Minimization (CoRM) whose objective ensures that the model predicts a class using its core features.
  arXiv  Detail & Related papers  (2022-03-28T01:53:34Z)
• A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes [58.633364000258645]
  We call this dataset RIVAL10 consisting of roughly $26k$ instances over $10$ classes. We evaluate the sensitivity of a broad set of models to noise corruptions in foregrounds, backgrounds and attributes. In our analysis, we consider diverse state-of-the-art architectures (ResNets, Transformers) and training procedures (CLIP, SimCLR, DeiT, Adversarial Training)
  arXiv  Detail & Related papers  (2022-01-26T06:31:28Z)
• Sparse MoEs meet Efficient Ensembles [49.313497379189315]
  We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs) We present Efficient Ensemble of Experts (E$3$), a scalable and simple ensemble of sparse MoEs that takes the best of both classes of models, while using up to 45% fewer FLOPs than a deep ensemble.
  arXiv  Detail & Related papers  (2021-10-07T11:58:35Z)
• CrossedWires: A Dataset of Syntactically Equivalent but Semantically Disparate Deep Learning Models [4.487528701954606]
  We present a living dataset of models and hyperparameters, called CrossedWires, that exposes semantic differences between two popular deep learning frameworks: PyTorch and Denseflow. The CrossedWires dataset provides an opportunity to study semantic differences between syntactically equivalent models across popular deep learning frameworks.
  arXiv  Detail & Related papers  (2021-08-29T07:27:16Z)
• ANNETTE: Accurate Neural Network Execution Time Estimation with Stacked Models [56.21470608621633]
  We propose a time estimation framework to decouple the architectural search from the target hardware. The proposed methodology extracts a set of models from micro- kernel and multi-layer benchmarks and generates a stacked model for mapping and network execution time estimation. We compare estimation accuracy and fidelity of the generated mixed models, statistical models with the roofline model, and a refined roofline model for evaluation.
  arXiv  Detail & Related papers  (2021-05-07T11:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.