Strengthening the Internal Adversarial Robustness in Lifted Neural Networks

• URL: http://arxiv.org/abs/2503.07818v1
• Date: Mon, 10 Mar 2025 20:00:38 GMT
• Title: Strengthening the Internal Adversarial Robustness in Lifted Neural Networks
• Authors: Christopher Zach,
• Abstract summary: We first investigate how adversarial robustness in this framework can be further strengthened by solely modifying the training loss.<n>In a second step we fix some remaining limitations and arrive at a novel training loss for lifted neural networks, that combines targeted and untargeted adversarial perturbations.
• Score: 9.781171732431169
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lifted neural networks (i.e. neural architectures explicitly optimizing over respective network potentials to determine the neural activities) can be combined with a type of adversarial training to gain robustness for internal as well as input layers, in addition to improved generalization performance. In this work we first investigate how adversarial robustness in this framework can be further strengthened by solely modifying the training loss. In a second step we fix some remaining limitations and arrive at a novel training loss for lifted neural networks, that combines targeted and untargeted adversarial perturbations.

Related papers

• Robust Spiking Neural Networks Against Adversarial Attacks [49.08210314590693]
  Spiking Neural Networks (SNNs) represent a promising paradigm for energy-efficient neuromorphic computing.<n>In this study, we theoretically demonstrate that threshold-neighboring spiking neurons are the key factors limiting the robustness of directly trained SNNs.<n>We find that these neurons set the upper limits for the maximum potential strength of adversarial attacks and are prone to state-flipping under minor disturbances.
  arXiv  Detail & Related papers  (2026-02-24T05:06:12Z)
• A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks [66.80058515743468]
  Training Spiking Neural Networks (SNNs) and Binary Neural Networks (BNNs) is challenging because of the non-differentiable spike generation function.<n>We present a novel perspective on the dynamics of SNNs and their close connection to BNNs through an analysis of the backpropagation process.<n>Specifically, we leverage a structure of multiple shortcuts and a knowledge distillation-based training technique to improve the training of (binary-weight) SNNs.
  arXiv  Detail & Related papers  (2025-08-18T04:11:06Z)
• Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning [57.3885832382455]
  We show that introducing static network sparsity alone can unlock further scaling potential beyond dense counterparts with state-of-the-art architectures.<n>Our analysis reveals that, in contrast to naively scaling up dense DRL networks, such sparse networks achieve both higher parameter efficiency for network expressivity.
  arXiv  Detail & Related papers  (2025-06-20T17:54:24Z)
• Training Verifiably Robust Agents Using Set-Based Reinforcement Learning [8.217552831952]
  We train neural networks utilizing entire sets of perturbed inputs and maximize the worst-case reward. The obtained agents are verifiably more robust than agents obtained by related work, making them more applicable in safety-critical environments.
  arXiv  Detail & Related papers  (2024-08-17T06:26:17Z)
• Compositional Curvature Bounds for Deep Neural Networks [7.373617024876726]
  A key challenge that threatens the widespread use of neural networks in safety-critical applications is their vulnerability to adversarial attacks. We study the second-order behavior of continuously differentiable deep neural networks, focusing on robustness against adversarial perturbations. We introduce a novel algorithm to analytically compute provable upper bounds on the second derivative of neural networks.
  arXiv  Detail & Related papers  (2024-06-07T17:50:15Z)
• Quantum-Inspired Analysis of Neural Network Vulnerabilities: The Role of Conjugate Variables in System Attacks [54.565579874913816]
  Neural networks demonstrate inherent vulnerability to small, non-random perturbations, emerging as adversarial attacks. A mathematical congruence manifests between this mechanism and the quantum physics' uncertainty principle, casting light on a hitherto unanticipated interdisciplinarity.
  arXiv  Detail & Related papers  (2024-02-16T02:11:27Z)
• Towards Improving Robustness Against Common Corruptions using Mixture of Class Specific Experts [10.27974860479791]
  This paper introduces a novel paradigm known as the Mixture of Class-Specific Expert Architecture. The proposed architecture aims to mitigate vulnerabilities associated with common neural network structures.
  arXiv  Detail & Related papers  (2023-11-16T20:09:47Z)
• Learning Dynamics and Generalization in Reinforcement Learning [59.530058000689884]
  We show theoretically that temporal difference learning encourages agents to fit non-smooth components of the value function early in training. We show that neural networks trained using temporal difference algorithms on dense reward tasks exhibit weaker generalization between states than randomly networks and gradient networks trained with policy methods.
  arXiv  Detail & Related papers  (2022-06-05T08:49:16Z)
• Defensive Tensorization [113.96183766922393]
  We propose tensor defensiveization, an adversarial defence technique that leverages a latent high-order factorization of the network. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We validate the versatility of our approach across domains and low-precision architectures by considering an audio task and binary networks.
  arXiv  Detail & Related papers  (2021-10-26T17:00:16Z)
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks [98.21130211336964]
  Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. In this paper, we investigate the impact of network width and depth on the robustness of adversarially trained DNNs.
  arXiv  Detail & Related papers  (2021-10-07T23:13:33Z)
• ASAT: Adaptively Scaled Adversarial Training in Time Series [21.65050910881857]
  We take the first step to introduce adversarial training in time series analysis by taking the finance field as an example. We propose the adaptively scaled adversarial training (ASAT) in time series analysis, by treating data at different time slots with time-dependent importance weights. Experimental results show that the proposed ASAT can improve both the accuracy and the adversarial robustness of neural networks.
  arXiv  Detail & Related papers  (2021-08-20T03:13:34Z)
• Clusterability in Neural Networks [9.190168301432811]
  We find that a trained neural network is typically more clusterable than randomly networks, and often clusterable relative to random networks with the same distribution of weights. We also exhibit novel methods to promote clusterability in neural network training, and find that in multi-layer perceptrons they lead to more clusterable networks with little reduction in accuracy.
  arXiv  Detail & Related papers  (2021-03-04T23:53:53Z)
• Feature Purification: How Adversarial Training Performs Robust Deep Learning [66.05472746340142]
  We show a principle that we call Feature Purification, where we show one of the causes of the existence of adversarial examples is the accumulation of certain small dense mixtures in the hidden weights during the training process of a neural network. We present both experiments on the CIFAR-10 dataset to illustrate this principle, and a theoretical result proving that for certain natural classification tasks, training a two-layer neural network with ReLU activation using randomly gradient descent indeed this principle.
  arXiv  Detail & Related papers  (2020-05-20T16:56:08Z)
• Learn2Perturb: an End-to-end Feature Perturbation Learning to Improve Adversarial Robustness [79.47619798416194]
  Learn2Perturb is an end-to-end feature perturbation learning approach for improving the adversarial robustness of deep neural networks. Inspired by the Expectation-Maximization, an alternating back-propagation training algorithm is introduced to train the network and noise parameters consecutively.
  arXiv  Detail & Related papers  (2020-03-02T18:27:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.