Are Latent Vulnerabilities Hidden Gems for Software Vulnerability Prediction? An Empirical Study

• URL: http://arxiv.org/abs/2401.11105v1
• Date: Sat, 20 Jan 2024 03:36:01 GMT
• Title: Are Latent Vulnerabilities Hidden Gems for Software Vulnerability Prediction? An Empirical Study
• Authors: Triet H. M. Le, Xiaoning Du, M. Ali Babar
• Abstract summary: latent vulnerable functions can increase the number of SVs by 4x on average and correct up to 5k mislabeled functions. Despite the noise, we show that the state-of-the-art SV prediction model can significantly benefit from such latent SVs.
• Score: 4.830367174383139
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Collecting relevant and high-quality data is integral to the development of effective Software Vulnerability (SV) prediction models. Most of the current SV datasets rely on SV-fixing commits to extract vulnerable functions and lines. However, none of these datasets have considered latent SVs existing between the introduction and fix of the collected SVs. There is also little known about the usefulness of these latent SVs for SV prediction. To bridge these gaps, we conduct a large-scale study on the latent vulnerable functions in two commonly used SV datasets and their utilization for function-level and line-level SV predictions. Leveraging the state-of-the-art SZZ algorithm, we identify more than 100k latent vulnerable functions in the studied datasets. We find that these latent functions can increase the number of SVs by 4x on average and correct up to 5k mislabeled functions, yet they have a noise level of around 6%. Despite the noise, we show that the state-of-the-art SV prediction model can significantly benefit from such latent SVs. The improvements are up to 24.5% in the performance (F1-Score) of function-level SV predictions and up to 67% in the effectiveness of localizing vulnerable lines. Overall, our study presents the first promising step toward the use of latent SVs to improve the quality of SV datasets and enhance the performance of SV prediction tasks.

Related papers

• VLRewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models [66.56298924208319]
  Vision-language generative reward models (VL-GenRMs) play a crucial role in aligning and evaluating multimodal AI systems. Current assessment methods rely on AI-annotated preference labels from traditional tasks. We introduce VL-RewardBench, a benchmark spanning general multimodal queries, visual hallucination detection, and complex reasoning tasks.
  arXiv  Detail & Related papers  (2024-11-26T14:08:34Z)
• Automatic Data Labeling for Software Vulnerability Prediction Models: How Far Are We? [0.0]
  Software Vulnerability (SV) prediction needs large-sized and high-quality data to perform well. We quantitatively and qualitatively study the quality and use of the state-of-the-art auto-labeled SV data, D2A, for SV prediction.
  arXiv  Detail & Related papers  (2024-07-25T06:22:25Z)
• Mitigating Data Imbalance for Software Vulnerability Assessment: Does Data Augmentation Help? [0.0]
  We show that mitigating data imbalance can significantly improve the predictive performance of models for all the Common Vulnerability Scoring System (CVSS) tasks. We also discover that simple text augmentation like combining random text insertion, deletion, and replacement can outperform the baseline across the board.
  arXiv  Detail & Related papers  (2024-07-15T13:47:55Z)
• Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving [55.93813178692077]
  We present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms. We assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction. Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data.
  arXiv  Detail & Related papers  (2024-05-27T17:59:39Z)
• Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPT [0.0]
  We conduct an empirical study to evaluate the impact of SV data scarcity in emerging languages on the state-of-the-art SV prediction model. We train and test the state-of-the-art model based on CodeBERT with and without data sampling techniques for function-level and line-level SV prediction.
  arXiv  Detail & Related papers  (2024-04-26T01:57:12Z)
• What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases [87.65903426052155]
  We perform a large-scale transfer learning experiment aimed at discovering latent vision-language skills from data. We show that generation tasks suffer from a length bias, suggesting benchmarks should balance tasks with varying output lengths. We present a new dataset, OLIVE, which simulates user instructions in the wild and presents challenges dissimilar to all datasets we tested.
  arXiv  Detail & Related papers  (2024-04-03T02:40:35Z)
• Improving the Adversarial Robustness of NLP Models by Information Bottleneck [112.44039792098579]
  Non-robust features can be easily manipulated by adversaries to fool NLP models. In this study, we explore the feasibility of capturing task-specific robust features, while eliminating the non-robust ones by using the information bottleneck theory. We show that the models trained with our information bottleneck-based method are able to achieve a significant improvement in robust accuracy.
  arXiv  Detail & Related papers  (2022-06-11T12:12:20Z)
• On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models [0.0]
  We use large-scale data from 1,782 functions of 429 SVs in 200 real-world projects to develop Machine Learning models for function-level SV assessment tasks. We show that vulnerable statements are 5.8 times smaller in size, yet exhibit 7.5-114.5% stronger assessment performance.
  arXiv  Detail & Related papers  (2022-03-16T06:29:40Z)
• VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements [62.93814803258067]
  This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements in source code. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively.
  arXiv  Detail & Related papers  (2021-12-20T22:45:27Z)
• A Survey on Data-driven Software Vulnerability Assessment and Prioritization [0.0]
  Software Vulnerabilities (SVs) are increasing in complexity and scale, posing great security risks to many software systems. Data-driven techniques such as Machine Learning and Deep Learning have taken SV assessment and prioritization to the next level.
  arXiv  Detail & Related papers  (2021-07-18T04:49:22Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.