Being Patient and Persistent: Optimizing An Early Stopping Strategy for Deep Learning in Profiled Attacks

• URL: http://arxiv.org/abs/2111.14416v1
• Date: Mon, 29 Nov 2021 09:54:45 GMT
• Title: Being Patient and Persistent: Optimizing An Early Stopping Strategy for Deep Learning in Profiled Attacks
• Authors: Servio Paguada, Lejla Batina, Ileana Buhan, Igor Armendariz
• Abstract summary: We propose an early stopping algorithm that reliably recognizes the model's optimal state during training. We formalize two conditions, persistence and patience, for a deep learning model to be optimal.
• Score: 2.7748013252318504
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The absence of an algorithm that effectively monitors deep learning models used in side-channel attacks increases the difficulty of evaluation. If the attack is unsuccessful, the question is if we are dealing with a resistant implementation or a faulty model. We propose an early stopping algorithm that reliably recognizes the model's optimal state during training. The novelty of our solution is an efficient implementation of guessing entropy estimation. Additionally, we formalize two conditions, persistence and patience, for a deep learning model to be optimal. As a result, the model converges with fewer traces.

Related papers

• Ranked Entropy Minimization for Continual Test-Time Adaptation [7.5140668729696145]
  Test-time adaptation aims to adapt to realistic environments in an online manner by learning during test time.<n>Entropy minimization has emerged as a principal strategy for test-time adaptation due to its efficiency and adaptability.<n>We propose ranked entropy minimization to mitigate the stability problem of the entropy minimization method.
  arXiv  Detail & Related papers  (2025-05-22T09:29:38Z)
• Deep Minimax Classifiers for Imbalanced Datasets with a Small Number of Minority Samples [5.217870815854702]
  We propose a novel minimax learning algorithm designed to minimize the risk of worst-performing classes. Our proposed algorithm has a provable convergence property, and empirical results indicate that our algorithm performs better than or is comparable to existing methods.
  arXiv  Detail & Related papers  (2025-02-24T08:20:02Z)
• Accelerated zero-order SGD under high-order smoothness and overparameterized regime [79.85163929026146]
  We present a novel gradient-free algorithm to solve convex optimization problems. Such problems are encountered in medicine, physics, and machine learning. We provide convergence guarantees for the proposed algorithm under both types of noise.
  arXiv  Detail & Related papers  (2024-11-21T10:26:17Z)
• Keeping Deep Learning Models in Check: A History-Based Approach to Mitigate Overfitting [18.952459066212523]
  Overfitting affects the quality, reliability, and trustworthiness of software systems that utilize deep learning models. We propose a simple, yet powerful approach that can both detect and prevent overfitting based on the training history. Our approach achieves an F1 score of 0.91 which is at least 5% higher than the current best-performing non-intrusive overfitting detection approach.
  arXiv  Detail & Related papers  (2024-01-18T19:56:27Z)
• Adaptive Sparse Gaussian Process [0.0]
  We propose the first adaptive sparse Gaussian Process (GP) able to address all these issues. We first reformulate a variational sparse GP algorithm to make it adaptive through a forgetting factor. We then propose updating a single inducing point of the sparse GP model together with the remaining model parameters every time a new sample arrives.
  arXiv  Detail & Related papers  (2023-02-20T21:34:36Z)
• Constrained Online Two-stage Stochastic Optimization: Near Optimal Algorithms via Adversarial Learning [1.994307489466967]
  We consider an online two-stage optimization with long-term constraints over a finite horizon of $T$ periods. We develop online algorithms for the online two-stage problem from adversarial learning algorithms.
  arXiv  Detail & Related papers  (2023-02-02T10:33:09Z)
• Oracle Inequalities for Model Selection in Offline Reinforcement Learning [105.74139523696284]
  We study the problem of model selection in offline RL with value function approximation. We propose the first model selection algorithm for offline RL that achieves minimax rate-optimal inequalities up to logarithmic factors. We conclude with several numerical simulations showing it is capable of reliably selecting a good model class.
  arXiv  Detail & Related papers  (2022-11-03T17:32:34Z)
• Uncertainty Estimation for Language Reward Models [5.33024001730262]
  Language models can learn a range of capabilities from unsupervised training on text corpora. It is often easier for humans to choose between options than to provide labeled data, and prior work has achieved state-of-the-art performance by training a reward model from such preference comparisons. We seek to address these problems via uncertainty estimation, which can improve sample efficiency and robustness using active learning and risk-averse reinforcement learning.
  arXiv  Detail & Related papers  (2022-03-14T20:13:21Z)
• On Optimal Early Stopping: Over-informative versus Under-informative Parametrization [13.159777131162961]
  We develop theoretical results to reveal the relationship between the optimal early stopping time and model dimension. We demonstrate experimentally that our theoretical results on optimal early stopping time corresponds to the training process of deep neural networks.
  arXiv  Detail & Related papers  (2022-02-20T18:20:06Z)
• Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization [60.72410937614299]
  We propose a new tractable bi-level optimization problem, design and analyze a new set of algorithms termed Bi-level AT (FAST-BAT) FAST-BAT is capable of defending sign-based projected descent (PGD) attacks without calling any gradient sign method and explicit robust regularization.
  arXiv  Detail & Related papers  (2021-12-23T06:25:36Z)
• Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits [55.740716446995805]
  We study a novel attack paradigm, which modifies model parameters in the deployment stage for malicious purposes. Our goal is to misclassify a specific sample into a target class without any sample modification. By utilizing the latest technique in integer programming, we equivalently reformulate this BIP problem as a continuous optimization problem.
  arXiv  Detail & Related papers  (2021-02-21T03:13:27Z)
• Outlier-Robust Learning of Ising Models Under Dobrushin's Condition [57.89518300699042]
  We study the problem of learning Ising models satisfying Dobrushin's condition in the outlier-robust setting where a constant fraction of the samples are adversarially corrupted. Our main result is to provide the first computationally efficient robust learning algorithm for this problem with near-optimal error guarantees.
  arXiv  Detail & Related papers  (2021-02-03T18:00:57Z)
• Model-Augmented Actor-Critic: Backpropagating through Paths [81.86992776864729]
  Current model-based reinforcement learning approaches use the model simply as a learned black-box simulator. We show how to make more effective use of the model by exploiting its differentiability.
  arXiv  Detail & Related papers  (2020-05-16T19:18:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.