Towards Causal Deep Learning for Vulnerability Detection
- URL: http://arxiv.org/abs/2310.07958v5
- Date: Mon, 15 Jan 2024 04:21:08 GMT
- Title: Towards Causal Deep Learning for Vulnerability Detection
- Authors: Md Mahbubur Rahman, Ira Ceka, Chengzhi Mao, Saikat Chakraborty,
Baishakhi Ray, and Wei Le
- Abstract summary: We introduce do calculus based causal learning to software engineering models.
Our results show that CausalVul consistently improved the model accuracy, robustness and OOD performance.
- Score: 31.59558109518435
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep learning vulnerability detection has shown promising results in recent
years. However, an important challenge that still blocks it from being very
useful in practice is that the model is not robust under perturbation and it
cannot generalize well over the out-of-distribution (OOD) data, e.g., applying
a trained model to unseen projects in real world. We hypothesize that this is
because the model learned non-robust features, e.g., variable names, that have
spurious correlations with labels. When the perturbed and OOD datasets no
longer have the same spurious features, the model prediction fails. To address
the challenge, in this paper, we introduced causality into deep learning
vulnerability detection. Our approach CausalVul consists of two phases. First,
we designed novel perturbations to discover spurious features that the model
may use to make predictions. Second, we applied the causal learning algorithms,
specifically, do-calculus, on top of existing deep learning models to
systematically remove the use of spurious features and thus promote causal
based prediction. Our results show that CausalVul consistently improved the
model accuracy, robustness and OOD performance for all the state-of-the-art
models and datasets we experimented. To the best of our knowledge, this is the
first work that introduces do calculus based causal learning to software
engineering models and shows it's indeed useful for improving the model
accuracy, robustness and generalization. Our replication package is located at
https://figshare.com/s/0ffda320dcb96c249ef2.
Related papers
- Root Causing Prediction Anomalies Using Explainable AI [3.970146574042422]
We present a novel application of explainable AI (XAI) for root-causing performance degradation in machine learning models.
A single feature corruption can cause cascading feature, label and concept drifts.
We have successfully applied this technique to improve the reliability of models used in personalized advertising.
arXiv Detail & Related papers (2024-03-04T19:38:50Z) - Three-Stage Adjusted Regression Forecasting (TSARF) for Software Defect
Prediction [5.826476252191368]
Nonhomogeneous Poisson process (NHPP) SRGM are the most commonly employed models.
Increased model complexity presents a challenge in identifying robust and computationally efficient algorithms.
arXiv Detail & Related papers (2024-01-31T02:19:35Z) - Stubborn Lexical Bias in Data and Models [50.79738900885665]
We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data.
We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations.
Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
arXiv Detail & Related papers (2023-06-03T20:12:27Z) - Preserving Knowledge Invariance: Rethinking Robustness Evaluation of
Open Information Extraction [50.62245481416744]
We present the first benchmark that simulates the evaluation of open information extraction models in the real world.
We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique.
By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
arXiv Detail & Related papers (2023-05-23T12:05:09Z) - An Empirical Study of Deep Learning Models for Vulnerability Detection [4.243592852049963]
We surveyed and reproduced 9 state-of-the-art deep learning models on 2 widely used vulnerability detection datasets.
We investigated model capabilities, training data, and model interpretation.
Our findings can help better understand model results, provide guidance on preparing training data, and improve the robustness of the models.
arXiv Detail & Related papers (2022-12-15T19:49:34Z) - KL-divergence Based Deep Learning for Discrete Time Model [12.165326681174408]
We develop a Kullback-Leibler-based (KL) deep learning procedure to integrate external survival prediction models with newly collected time-to-event data.
Time-dependent KL discrimination information is utilized to measure the discrepancy between the external and internal data.
arXiv Detail & Related papers (2022-08-10T01:46:26Z) - On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
Differentiable causal discovery has proposed to factorize the data generating process into a set of modules.
We study the generalization and adaption performance of such modular neural causal models.
Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
arXiv Detail & Related papers (2022-06-09T17:12:32Z) - Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators.
They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions.
We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
arXiv Detail & Related papers (2021-02-22T07:02:37Z) - Anomaly Detection of Time Series with Smoothness-Inducing Sequential
Variational Auto-Encoder [59.69303945834122]
We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series.
Our model parameterizes mean and variance for each time-stamp with flexible neural networks.
We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
arXiv Detail & Related papers (2021-02-02T06:15:15Z) - Amortized learning of neural causal representations [10.140457813764554]
Causal models can compactly and efficiently encode the data-generating process under all interventions.
These models are often represented as Bayesian networks and learning them scales poorly with the number of variables.
We represent a novel algorithm called textitcausal relational networks (CRN) for learning causal models using neural networks.
arXiv Detail & Related papers (2020-08-21T04:35:06Z) - Learning Causal Models Online [103.87959747047158]
Predictive models can rely on spurious correlations in the data for making predictions.
One solution for achieving strong generalization is to incorporate causal structures in the models.
We propose an online algorithm that continually detects and removes spurious features.
arXiv Detail & Related papers (2020-06-12T20:49:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.