On Robustness and Generalization of ML-Based Congestion Predictors to
Valid and Imperceptible Perturbations
- URL: http://arxiv.org/abs/2403.00103v1
- Date: Thu, 29 Feb 2024 20:11:47 GMT
- Title: On Robustness and Generalization of ML-Based Congestion Predictors to
Valid and Imperceptible Perturbations
- Authors: Chester Holtz, Yucheng Wang, Chung-Kuan Cheng, Bill Lin
- Abstract summary: Recent work has demonstrated that neural networks are generally vulnerable to small, carefully chosen perturbations of their input.
We show that state-of-the-art CNN and GNN-based congestion models exhibit brittleness to imperceptible perturbations.
Our work indicates that CAD engineers should be cautious when integrating neural network-based mechanisms in EDA flows.
- Score: 9.982978359852494
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: There is substantial interest in the use of machine learning (ML)-based
techniques throughout the electronic computer-aided design (CAD) flow,
particularly methods based on deep learning. However, while deep learning
methods have achieved state-of-the-art performance in several applications,
recent work has demonstrated that neural networks are generally vulnerable to
small, carefully chosen perturbations of their input (e.g. a single pixel
change in an image). In this work, we investigate robustness in the context of
ML-based EDA tools -- particularly for congestion prediction. As far as we are
aware, we are the first to explore this concept in the context of ML-based EDA.
We first describe a novel notion of imperceptibility designed specifically
for VLSI layout problems defined on netlists and cell placements. Our
definition of imperceptibility is characterized by a guarantee that a
perturbation to a layout will not alter its global routing. We then demonstrate
that state-of-the-art CNN and GNN-based congestion models exhibit brittleness
to imperceptible perturbations. Namely, we show that when a small number of
cells (e.g. 1%-5% of cells) have their positions shifted such that a measure of
global congestion is guaranteed to remain unaffected (e.g. 1% of the design
adversarially shifted by 0.001% of the layout space results in a predicted
decrease in congestion of up to 90%, while no change in congestion is implied
by the perturbation). In other words, the quality of a predictor can be made
arbitrarily poor (i.e. can be made to predict that a design is
"congestion-free") for an arbitrary input layout. Next, we describe a simple
technique to train predictors that improves robustness to these perturbations.
Our work indicates that CAD engineers should be cautious when integrating
neural network-based mechanisms in EDA flows to ensure robust and high-quality
results.
Related papers
- The Over-Certainty Phenomenon in Modern UDA Algorithms [3.301728339780329]
We aim to recalibrate a neural network's decision boundaries in relation to its cognizance of the data it observes.
We propose a UDA algorithm that not only augments accuracy but also assures model calibration.
arXiv Detail & Related papers (2024-04-24T19:55:50Z) - Rethinking PGD Attack: Is Sign Function Necessary? [131.6894310945647]
We present a theoretical analysis of how such sign-based update algorithm influences step-wise attack performance.
We propose a new raw gradient descent (RGD) algorithm that eliminates the use of sign.
The effectiveness of the proposed RGD algorithm has been demonstrated extensively in experiments.
arXiv Detail & Related papers (2023-12-03T02:26:58Z) - Active Inference on the Edge: A Design Study [5.815300670677979]
Active Inference (ACI) is a concept from neuroscience that describes how the brain constantly predicts and evaluates sensory information to decrease long-term surprise.
We show how our ACI agent was able to quickly and traceably solve an optimization problem while fulfilling requirements.
arXiv Detail & Related papers (2023-11-17T16:03:04Z) - Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs.
We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD.
We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
arXiv Detail & Related papers (2023-10-02T03:25:32Z) - Bridging Precision and Confidence: A Train-Time Loss for Calibrating
Object Detection [58.789823426981044]
We propose a novel auxiliary loss formulation that aims to align the class confidence of bounding boxes with the accurateness of predictions.
Our results reveal that our train-time loss surpasses strong calibration baselines in reducing calibration error for both in and out-domain scenarios.
arXiv Detail & Related papers (2023-03-25T08:56:21Z) - A Simple Approach to Improve Single-Model Deep Uncertainty via
Distance-Awareness [33.09831377640498]
We study approaches to improve uncertainty property of a single network, based on a single, deterministic representation.
We propose Spectral-normalized Neural Gaussian Process (SNGP), a simple method that improves the distance-awareness ability of modern DNNs.
On a suite of vision and language understanding benchmarks, SNGP outperforms other single-model approaches in prediction, calibration and out-of-domain detection.
arXiv Detail & Related papers (2022-05-01T05:46:13Z) - Unveiling the potential of Graph Neural Networks for robust Intrusion
Detection [2.21481607673149]
We propose a novel Graph Neural Network (GNN) model to learn flow patterns of attacks structured as graphs.
Our model is able to maintain the same level of accuracy as in previous experiments, while state-of-the-art ML techniques degrade up to 50% their accuracy (F1-score) under adversarial attacks.
arXiv Detail & Related papers (2021-07-30T16:56:39Z) - TELESTO: A Graph Neural Network Model for Anomaly Classification in
Cloud Services [77.454688257702]
Machine learning (ML) and artificial intelligence (AI) are applied on IT system operation and maintenance.
One direction aims at the recognition of re-occurring anomaly types to enable remediation automation.
We propose a method that is invariant to dimensionality changes of given data.
arXiv Detail & Related papers (2021-02-25T14:24:49Z) - Attribute-Guided Adversarial Training for Robustness to Natural
Perturbations [64.35805267250682]
We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space.
Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
arXiv Detail & Related papers (2020-12-03T10:17:30Z) - Relaxing the Constraints on Predictive Coding Models [62.997667081978825]
Predictive coding is an influential theory of cortical function which posits that the principal computation the brain performs is the minimization of prediction errors.
Standard implementations of the algorithm still involve potentially neurally implausible features such as identical forward and backward weights, backward nonlinear derivatives, and 1-1 error unit connectivity.
In this paper, we show that these features are not integral to the algorithm and can be removed either directly or through learning additional sets of parameters with Hebbian update rules without noticeable harm to learning performance.
arXiv Detail & Related papers (2020-10-02T15:21:37Z) - Local error quantification for Neural Network Differential Equation
solvers [6.09170287691728]
We develop methods that allow NN DE solvers to be more accurate and efficient.
We achieve this via methods that can precisely estimate NN DE solver prediction errors point-wise.
We exemplify the utility of our methods by testing them on a nonlinear and a chaotic system each.
arXiv Detail & Related papers (2020-08-24T08:22:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.