Breaking Boundaries: Balancing Performance and Robustness in Deep
Wireless Traffic Forecasting
- URL: http://arxiv.org/abs/2311.09790v3
- Date: Tue, 28 Nov 2023 15:53:00 GMT
- Title: Breaking Boundaries: Balancing Performance and Robustness in Deep
Wireless Traffic Forecasting
- Authors: Romain Ilbert, Thai V. Hoang, Zonghua Zhang, Themis Palpanas
- Abstract summary: Balancing the trade-off between accuracy and robustness is a long-standing challenge in time series forecasting.
We study a wide array of perturbation scenarios and propose novel defense mechanisms against adversarial attacks using real-world telecom data.
- Score: 11.029214459961114
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Balancing the trade-off between accuracy and robustness is a long-standing
challenge in time series forecasting. While most of existing robust algorithms
have achieved certain suboptimal performance on clean data, sustaining the same
performance level in the presence of data perturbations remains extremely hard.
In this paper, we study a wide array of perturbation scenarios and propose
novel defense mechanisms against adversarial attacks using real-world telecom
data. We compare our strategy against two existing adversarial training
algorithms under a range of maximal allowed perturbations, defined using
$\ell_{\infty}$-norm, $\in [0.1,0.4]$. Our findings reveal that our hybrid
strategy, which is composed of a classifier to detect adversarial examples, a
denoiser to eliminate noise from the perturbed data samples, and a standard
forecaster, achieves the best performance on both clean and perturbed data. Our
optimal model can retain up to $92.02\%$ the performance of the original
forecasting model in terms of Mean Squared Error (MSE) on clean data, while
being more robust than the standard adversarially trained models on perturbed
data. Its MSE is 2.71$\times$ and 2.51$\times$ lower than those of comparing
methods on normal and perturbed data, respectively. In addition, the components
of our models can be trained in parallel, resulting in better computational
efficiency. Our results indicate that we can optimally balance the trade-off
between the performance and robustness of forecasting models by improving the
classifier and denoiser, even in the presence of sophisticated and destructive
poisoning attacks.
Related papers
- PUMA: margin-based data pruning [51.12154122266251]
We focus on data pruning, where some training samples are removed based on the distance to the model classification boundary (i.e., margin)
We propose PUMA, a new data pruning strategy that computes the margin using DeepFool.
We show that PUMA can be used on top of the current state-of-the-art methodology in robustness, and it is able to significantly improve the model performance unlike the existing data pruning strategies.
arXiv Detail & Related papers (2024-05-10T08:02:20Z) - Advancing the Robustness of Large Language Models through Self-Denoised Smoothing [50.54276872204319]
Large language models (LLMs) have achieved significant success, but their vulnerability to adversarial perturbations has raised considerable concerns.
We propose to leverage the multitasking nature of LLMs to first denoise the noisy inputs and then to make predictions based on these denoised versions.
Unlike previous denoised smoothing techniques in computer vision, which require training a separate model to enhance the robustness of LLMs, our method offers significantly better efficiency and flexibility.
arXiv Detail & Related papers (2024-04-18T15:47:00Z) - A Systematic Approach to Robustness Modelling for Deep Convolutional
Neural Networks [0.294944680995069]
Recent work raises questions about the ability for even larger models to generalize to data outside of the controlled train and test sets.
We provide a method that uses induced failures to model the probability of failure as a function of time.
We examine the various trade-offs between cost, robustness, latency, and reliability to find that larger models do not significantly aid in adversarial robustness.
arXiv Detail & Related papers (2024-01-24T19:12:37Z) - Robust Trajectory Prediction against Adversarial Attacks [84.10405251683713]
Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving systems.
These methods are vulnerable to adversarial attacks, leading to serious consequences such as collisions.
In this work, we identify two key ingredients to defend trajectory prediction models against adversarial attacks.
arXiv Detail & Related papers (2022-07-29T22:35:05Z) - Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions.
In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data.
We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
arXiv Detail & Related papers (2022-01-11T23:01:12Z) - Fixing Data Augmentation to Improve Adversarial Robustness [21.485435979018256]
Adversarial training suffers from robust overfitting, a phenomenon where the robust test accuracy starts to decrease during training.
In this paper, we focus on both adversarials-driven and data-driven augmentations as a means to reduce robust overfitting.
We show large absolute improvements of +7.06% and +5.88% in robust accuracy compared to previous state-of-the-art methods.
arXiv Detail & Related papers (2021-03-02T18:58:33Z) - Adversarially Robust Classifier with Covariate Shift Adaptation [25.39995678746662]
Existing adversarially trained models typically perform inference on test examples independently from each other.
We show that simple adaptive batch normalization (BN) technique can significantly improve the robustness of these models for any random perturbations.
We further demonstrate that adaptive BN technique significantly improves robustness against common corruptions, while often enhancing performance against adversarial attacks.
arXiv Detail & Related papers (2021-02-09T19:51:56Z) - Robust Bayesian Inference for Discrete Outcomes with the Total Variation
Distance [5.139874302398955]
Models of discrete-valued outcomes are easily misspecified if the data exhibit zero-inflation, overdispersion or contamination.
Here, we introduce a robust discrepancy-based Bayesian approach using the Total Variation Distance (TVD)
We empirically demonstrate that our approach is robust and significantly improves predictive performance on a range of simulated and real world data.
arXiv Detail & Related papers (2020-10-26T09:53:06Z) - Evaluating Prediction-Time Batch Normalization for Robustness under
Covariate Shift [81.74795324629712]
We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift.
We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness.
The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
arXiv Detail & Related papers (2020-06-19T05:08:43Z) - Consistency Regularization for Certified Robustness of Smoothed
Classifiers [89.72878906950208]
A recent technique of randomized smoothing has shown that the worst-case $ell$-robustness can be transformed into the average-case robustness.
We found that the trade-off between accuracy and certified robustness of smoothed classifiers can be greatly controlled by simply regularizing the prediction consistency over noise.
arXiv Detail & Related papers (2020-06-07T06:57:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.