LightSC: The Making of a Usable Security Classification Tool for DevSecOps

• URL: http://arxiv.org/abs/2410.01762v1
• Date: Wed, 2 Oct 2024 17:17:14 GMT
• Title: LightSC: The Making of a Usable Security Classification Tool for DevSecOps
• Abstract summary: We propose five principles for a security classification to be emphDevOps-ready We then exemplify how one can make a security classification methodology DevOps-ready. Since such work seems to be new within the usable security community, we extract from our process a general, three-steps recipe' Our tool is perceived (by the test subjects) as most useful in the design phase, but also during the testing phase where the security class would be one of the metrics used to evaluate the quality of their software.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: DevSecOps, as the extension of DevOps with security training and tools, has become a popular way of developing modern software, especially in the Internet of Things arena, due to its focus on rapid development, with short release cycles, involving the user/client very closely. Security classification methods, on the other hand, are heavy and slow processes that require high expertise in security, the same as in other similar areas such as risk analysis or certification. As such, security classification methods are hardly compatible with the DevSecOps culture, which to the contrary, has moved away from the traditional style of penetration testing done only when the software product is in the final stages or already deployed. In this work, we first propose five principles for a security classification to be \emph{DevOps-ready}, two of which will be the focus for the rest of the paper, namely to be tool-based and easy to use for non-security experts, such as ordinary developers or system architects. We then exemplify how one can make a security classification methodology DevOps-ready. We do this through an interaction design process, where we create and evaluate the usability of a tool implementing the chosen methodology. Since such work seems to be new within the usable security community, and even more so in the software development (DevOps) community, we extract from our process a general, three-steps `recipe' that others can follow when making their own security methodologies DevOps-ready. The tool that we build is in itself a contribution of this process, as it can be independently used, extended, and/or integrated by developer teams into their DevSecOps tool-chains. Our tool is perceived (by the test subjects) as most useful in the design phase, but also during the testing phase where the security class would be one of the metrics used to evaluate the quality of their software.

Related papers

• HazardAuditor: From Executable Threats to Safer Computer-Use Agents [50.06351661912873]
  Existing guard models target static prompts and responses.<n>Existing safety platforms produce evaluation verdicts rather than the normalized supervision a guard model needs to learn.<n>We introduce HazardAuditor, an execution-grounded framework that closes both gaps.
  arXiv  Detail & Related papers  (2026-09-14T07:09:33Z)
• How Do You Choose Your AI Component? An Interview Study of Secure AI Integration in Practice [3.3043425525629293]
  We conduct semistructured interviews with 22 software developers, architects, and AI practitioners about how they integrate AI components into their software.<n>Our analysis finds that practitioners' model selection is predominantly driven by functional criteria, including performance, accuracy, cost, and specific features.<n>We observe a consistent lack of security concern throughout the AI component integration process, with established software supply chain lessons overlooked or ignored.
  arXiv  Detail & Related papers  (2026-07-18T06:30:12Z)
• SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents [87.26967184869198]
  SeClaw is a framework that combines specification-driven security task synthesis with execution-based security evaluation for Autonomous agents.<n> benchmark covers risks arising from resources, user tasks, environments, and intrinsic agent behaviors.
  arXiv  Detail & Related papers  (2026-06-01T14:23:42Z)
• From Preventive to Reactive: How AI Coding Assistants Transform Developers' Security Awareness [3.2382588276702315]
  We conducted interviews with 15 professional software engineers and observed them completing security-relevant coding tasks with AI assistance.<n>We find that AI coding assistants reorganize rather than eliminate security thinking, shifting it from the act of writing code to the act of reviewing it.<n>This paper contributes a practice-grounded account of how AI-assisted development reshapes the human side of secure coding.
  arXiv  Detail & Related papers  (2026-05-22T01:07:15Z)
• Anticipating Adversary Behavior in DevSecOps Scenarios through Large Language Models [2.2192966937452376]
  This work proposes integrating the Security Chaos Engineering (SCE) methodology with a new LLM-based flow to automate the creation of attack defense trees.<n>This will enable teams to stay one step ahead of attackers and implement previously unconsidered defenses.<n>Further detailed information about the experiment performed, along with the steps to replicate it, can be found in the following repository.
  arXiv  Detail & Related papers  (2026-02-15T11:43:04Z)
• Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents [68.20752678837377]
  We propose a principled taxonomy that transforms single-turn harmful tasks into multi-turn attack sequences.<n>Using this taxonomy, we construct MT-AgentRisk, the first benchmark to evaluate multi-turn tool-using agent safety.<n>We propose ToolShield, a training-free, tool-agnostic, self-exploration defense.
  arXiv  Detail & Related papers  (2026-02-13T18:38:18Z)
• DeepSight: An All-in-One LM Safety Toolkit [49.27916339801296]
  We propose an open-source project, namely DeepSight, to practice a new safety evaluation-diagnosis integrated paradigm.<n>DeepSight is low-cost, reproducible, efficient, and highly scalable large-scale model safety evaluation project consisting of a evaluation toolkit DeepSafe and a diagnosis toolkit DeepScan.<n>By unifying task and data protocols, we build a connection between the two stages and transform safety evaluation from black-box to white-box insight.
  arXiv  Detail & Related papers  (2026-02-12T15:43:14Z)
• DevOps-Gym: Benchmarking AI Agents in Software DevOps Cycle [84.01703913780946]
  Handling the DevOps cycle in real-world software requires analyzing large-scale projects, understanding dynamic program behaviors, leveraging domain-specific tools, and making sequential decisions.<n>We introduce DevOps-Gym, the first end-to-end benchmark for evaluating AI agents across core DevOps tasks.
  arXiv  Detail & Related papers  (2026-01-27T18:43:46Z)
• Safety Factories - a Manifesto [1.1916129241436584]
  We need to bridge the disconnect in methods and tools between software development and safety engineering.<n>We advocate for safety factories, which integrate safety tooling and methods into software development pipelines.
  arXiv  Detail & Related papers  (2025-09-10T05:01:25Z)
• OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety [58.201189860217724]
  We introduce OpenAgentSafety, a comprehensive framework for evaluating agent behavior across eight critical risk categories.<n>Unlike prior work, our framework evaluates agents that interact with real tools, including web browsers, code execution environments, file systems, bash shells, and messaging platforms.<n>It combines rule-based analysis with LLM-as-judge assessments to detect both overt and subtle unsafe behaviors.
  arXiv  Detail & Related papers  (2025-07-08T16:18:54Z)
• Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation [52.83870601473094]
  Embodied agents exhibit immense potential across a multitude of domains. Existing research predominantly concentrates on the security of general large language models. This paper introduces a novel input moderation framework, meticulously designed to safeguard embodied agents.
  arXiv  Detail & Related papers  (2025-04-22T08:34:35Z)
• Integrating DAST in Kanban and CI/CD: A Real World Security Case Study [2.3480418671346164]
  Web application attacks and exploited vulnerabilities are rising. It is increasingly crucial to integrate security into modern development practices. It is challenging to adopt security practices and activities in modern development practices.
  arXiv  Detail & Related papers  (2025-03-27T19:46:05Z)
• AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement [73.0700818105842]
  We introduce AISafetyLab, a unified framework and toolkit that integrates representative attack, defense, and evaluation methodologies for AI safety. AISafetyLab features an intuitive interface that enables developers to seamlessly apply various techniques. We conduct empirical studies on Vicuna, analyzing different attack and defense strategies to provide valuable insights into their comparative effectiveness.
  arXiv  Detail & Related papers  (2025-02-24T02:11:52Z)
• SecCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [47.11178028457252]
  We develop SecCodePLT, a unified and comprehensive evaluation platform for code GenAIs' risks. For insecure code, we introduce a new methodology for data creation that combines experts with automatic generation. For cyberattack helpfulness, we construct samples to prompt a model to generate actual attacks, along with dynamic metrics in our environment.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)
• Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? [60.84912551069379]
  We present the Code-Development Benchmark (Codev-Bench), a fine-grained, real-world, repository-level, and developer-centric evaluation framework. Codev-Agent is an agent-based system that automates repository crawling, constructs execution environments, extracts dynamic calling chains from existing unit tests, and generates new test samples to avoid data leakage.
  arXiv  Detail & Related papers  (2024-10-02T09:11:10Z)
• Hacking, The Lazy Way: LLM Augmented Pentesting [0.0]
  "LLM Augmented Pentesting" is demonstrated through a tool named "Pentest Copilot" Our research includes a "chain of thought" mechanism to streamline token usage and boost performance. We propose a novel file analysis approach, enabling LLMs to understand files.
  arXiv  Detail & Related papers  (2024-09-14T17:40:35Z)
• Continuous risk assessment in secure DevOps [0.24475591916185502]
  We argue how secure DevOps could profit from engaging with risk related activities within organisations. We focus on combining Risk Assessment (RA), particularly Threat Modelling (TM) and apply security considerations early in the software life-cycle.
  arXiv  Detail & Related papers  (2024-09-05T10:42:27Z)
• AI for DevSecOps: A Landscape and Future Opportunities [6.513361705307775]
  DevSecOps has emerged as one of the most rapidly evolving software development paradigms. With the growing concerns surrounding security in software systems, the DevSecOps paradigm has gained prominence. Integrating security into the DevOps workflow can impact agility and impede delivery speed.
  arXiv  Detail & Related papers  (2024-04-07T07:24:58Z)
• Teaching DevOps Security Education with Hands-on Labware: Automated Detection of Security Weakness in Python [4.280051038571455]
  We introduce hands-on learning modules that enable learners to be familiar with identifying known security weaknesses. To cultivate an engaging and motivating learning environment, our hands-on approach includes a pre-lab, hands-on and post lab sections.
  arXiv  Detail & Related papers  (2023-08-14T16:09:05Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Semantic Similarity-Based Clustering of Findings From Security Testing Tools [1.6058099298620423]
  In particular, it is common practice to use automated security testing tools that generate reports after inspecting a software artifact from multiple perspectives. To identify these duplicate findings manually, a security expert has to invest resources like time, effort, and knowledge. In this study, we investigated the potential of applying Natural Language Processing for clustering semantically similar security findings.
  arXiv  Detail & Related papers  (2022-11-20T19:03:19Z)
• Realistic simulation of users for IT systems in cyber ranges [63.20765930558542]
  We instrument each machine by means of an external agent to generate user activity. This agent combines both deterministic and deep learning based methods to adapt to different environment. We also propose conditional text generation models to facilitate the creation of conversations and documents.
  arXiv  Detail & Related papers  (2021-11-23T10:53:29Z)
• Empowered and Embedded: Ethics and Agile Processes [60.63670249088117]
  We argue that ethical considerations need to be embedded into the (agile) software development process. We put emphasis on the possibility to implement ethical deliberations in already existing and well established agile software development processes.
  arXiv  Detail & Related papers  (2021-07-15T11:14:03Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.