Security Defect Detection via Code Review: A Study of the OpenStack and
Qt Communities
- URL: http://arxiv.org/abs/2307.02326v1
- Date: Wed, 5 Jul 2023 14:30:41 GMT
- Title: Security Defect Detection via Code Review: A Study of the OpenStack and
Qt Communities
- Authors: Jiaxin Yu, Liming Fu, Peng Liang, Amjed Tahir, Mojtaba Shahin
- Abstract summary: Security defects are not prevalently discussed in code review.
More than half of the reviewers provided explicit fixing strategies/solutions to help developers fix security defects.
Disagreement between the developer and the reviewer are the main causes for not resolving security defects.
- Score: 7.2944322548786715
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Background: Despite the widespread use of automated security defect detection
tools, software projects still contain many security defects that could result
in serious damage. Such tools are largely context-insensitive and may not cover
all possible scenarios in testing potential issues, which makes them
susceptible to missing complex security defects. Hence, thorough detection
entails a synergistic cooperation between these tools and human-intensive
detection techniques, including code review. Code review is widely recognized
as a crucial and effective practice for identifying security defects. Aim: This
work aims to empirically investigate security defect detection through code
review. Method: To this end, we conducted an empirical study by analyzing code
review comments derived from four projects in the OpenStack and Qt communities.
Through manually checking 20,995 review comments obtained by keyword-based
search, we identified 614 comments as security-related. Results: Our results
show that (1) security defects are not prevalently discussed in code review,
(2) more than half of the reviewers provided explicit fixing
strategies/solutions to help developers fix security defects, (3) developers
tend to follow reviewers' suggestions and action the changes, (4) Not worth
fixing the defect now and Disagreement between the developer and the reviewer
are the main causes for not resolving security defects. Conclusions: Our
research results demonstrate that (1) software security practices should
combine manual code review with automated detection tools, achieving a more
comprehensive coverage to identifying and addressing security defects, and (2)
promoting appropriate standardization of practitioners' behaviors during code
review remains necessary for enhancing software security.
Related papers
- Fixing Smart Contract Vulnerabilities: A Comparative Analysis of
Literature and Developer's Practices [6.09162202256218]
We refer to vulnerability fixing in the ways found in the literature as guidelines.
It is not clear to what extent developers adhere to these guidelines, nor whether there are other viable common solutions and what they are.
The goal of our research is to fill knowledge gaps related to developers' observance of existing guidelines and to propose new and viable solutions to security vulnerabilities.
arXiv Detail & Related papers (2024-03-12T09:55:54Z) - Security Code Review by Large Language Models [9.309745288471374]
We conduct the first empirical study to understand the capabilities of Large Language Models (LLMs) in security code review.
We compare the performance of 6 LLMs under five different prompts with the state-of-the-art static analysis tools to detect and analyze security defects.
For the best-performing LLM, we conducted a linguistic analysis to explore quality problems in its responses.
arXiv Detail & Related papers (2024-01-29T17:13:44Z) - What Can Self-Admitted Technical Debt Tell Us About Security? A
Mixed-Methods Study [6.286506087629511]
Self-Admitted Technical Debt (SATD)
can be deemed as dreadful sources of information on potentially exploitable vulnerabilities and security flaws.
This work investigates the security implications of SATD from a technical and developer-centred perspective.
arXiv Detail & Related papers (2024-01-23T13:48:49Z) - A Survey and Comparative Analysis of Security Properties of CAN Authentication Protocols [92.81385447582882]
The Controller Area Network (CAN) bus leaves in-vehicle communications inherently non-secure.
This paper reviews and compares the 15 most prominent authentication protocols for the CAN bus.
We evaluate protocols based on essential operational criteria that contribute to ease of implementation.
arXiv Detail & Related papers (2024-01-19T14:52:04Z) - Toward Effective Secure Code Reviews: An Empirical Study of Security-Related Coding Weaknesses [14.134803943492345]
We conducted an empirical case study in two large open-source projects, OpenSSL and PHP.
Based on 135,560 code review comments, we found that reviewers raised security concerns in 35 out of 40 coding weakness categories.
Some coding weaknesses related to past vulnerabilities, such as memory errors and resource management, were discussed less often than the vulnerabilities.
arXiv Detail & Related papers (2023-11-28T00:49:00Z) - A Novel Approach to Identify Security Controls in Source Code [4.598579706242066]
This paper enumerates a comprehensive list of commonly used security controls and creates a dataset for each one of them.
It uses the state-of-the-art NLP technique Bidirectional Representations from Transformers (BERT) and the Tactic Detector from our prior work to show that security controls could be identified with high confidence.
arXiv Detail & Related papers (2023-07-10T21:14:39Z) - DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark.
We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions.
DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
arXiv Detail & Related papers (2023-07-04T01:34:41Z) - CodeLMSec Benchmark: Systematically Evaluating and Finding Security
Vulnerabilities in Black-Box Code Language Models [58.27254444280376]
Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks.
Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities.
This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure.
arXiv Detail & Related papers (2023-02-08T11:54:07Z) - Pre-trained Encoders in Self-Supervised Learning Improve Secure and
Privacy-preserving Supervised Learning [63.45532264721498]
Self-supervised learning is an emerging technique to pre-train encoders using unlabeled data.
We perform first systematic, principled measurement study to understand whether and when a pretrained encoder can address the limitations of secure or privacy-preserving supervised learning algorithms.
arXiv Detail & Related papers (2022-12-06T21:35:35Z) - Towards a Fair Comparison and Realistic Design and Evaluation Framework
of Android Malware Detectors [63.75363908696257]
We analyze 10 influential research works on Android malware detection using a common evaluation framework.
We identify five factors that, if not taken into account when creating datasets and designing detectors, significantly affect the trained ML models.
We conclude that the studied ML-based detectors have been evaluated optimistically, which justifies the good published results.
arXiv Detail & Related papers (2022-05-25T08:28:08Z) - Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance.
We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems.
We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
arXiv Detail & Related papers (2020-10-19T13:09:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.