Leveraging Large Language Models for Preliminary Security Risk Analysis: A Mission-Critical Case Study

• URL: http://arxiv.org/abs/2403.15756v1
• Date: Sat, 23 Mar 2024 07:59:30 GMT
• Title: Leveraging Large Language Models for Preliminary Security Risk Analysis: A Mission-Critical Case Study
• Authors: Matteo Esposito, Francesco Palagiano,
• Abstract summary: The speed and accuracy of human experts in PSRA significantly impact response time. No prior study has explored the capabilities of fine-tuned models (FTM) in PSRA. Our approach has proven successful in reducing errors in PSRA, hastening security risk detection, and minimizing false positives and negatives.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Preliminary security risk analysis (PSRA) provides a quick approach to identify, evaluate and propose remeditation to potential risks in specific scenarios. The extensive expertise required for an effective PSRA and the substantial ammount of textual-related tasks hinder quick assessments in mission-critical contexts, where timely and prompt actions are essential. The speed and accuracy of human experts in PSRA significantly impact response time. A large language model can quickly summarise information in less time than a human. To our knowledge, no prior study has explored the capabilities of fine-tuned models (FTM) in PSRA. Our case study investigates the proficiency of FTM to assist practitioners in PSRA. We manually curated 141 representative samples from over 50 mission-critical analyses archived by the industrial context team in the last five years.We compared the proficiency of the FTM versus seven human experts. Within the industrial context, our approach has proven successful in reducing errors in PSRA, hastening security risk detection, and minimizing false positives and negatives. This translates to cost savings for the company by averting unnecessary expenses associated with implementing unwarranted countermeasures. Therefore, experts can focus on more comprehensive risk analysis, leveraging LLMs for an effective preliminary assessment within a condensed timeframe.

Related papers

• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users. We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions. We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Risks and NLP Design: A Case Study on Procedural Document QA [52.557503571760215]
  We argue that clearer assessments of risks and harms to users will be possible when we specialize the analysis to more concrete applications and their plausible users. We conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
  arXiv  Detail & Related papers  (2024-08-16T17:23:43Z)
• Explainable Artificial Intelligence Techniques for Irregular Temporal Classification of Multidrug Resistance Acquisition in Intensive Care Unit Patients [7.727213847237959]
  This study introduces a novel methodology that integrates Gated Recurrent Units (GRUs) with advanced intrinsic and post-hoc interpretability techniques. Our methodology aims to identify specific risk factors associated with Multidrug-Resistant (MDR) infections in ICU patients.
  arXiv  Detail & Related papers  (2024-07-24T11:12:01Z)
• InferAct: Inferring Safe Actions for LLM-Based Agents Through Preemptive Evaluation and Human Feedback [70.54226917774933]
  This paper introduces InferAct, a novel approach to proactively detect potential errors before risky actions are executed. InferAct acts as a human proxy, detecting unsafe actions and alerting users for intervention. Experiments on three widely-used tasks demonstrate the effectiveness of InferAct.
  arXiv  Detail & Related papers  (2024-07-16T15:24:44Z)
• Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysis [7.098487130130114]
  Risk analysis principles are context-less. Risk analysis requires a vast knowledge of national and international regulations and standards. Large language models can quickly summarize information in less time than a human and can be fine-tuned to specific tasks.
  arXiv  Detail & Related papers  (2024-06-11T19:20:27Z)
• The Human Factor in Detecting Errors of Large Language Models: A Systematic Literature Review and Future Research Directions [0.0]
  Launch of ChatGPT by OpenAI in November 2022 marked a pivotal moment for Artificial Intelligence. Large Language Models (LLMs) demonstrate remarkable conversational capabilities across various domains. These models are susceptible to errors - "hallucinations" and omissions, generating incorrect or incomplete information.
  arXiv  Detail & Related papers  (2024-03-13T21:39:39Z)
• Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science [65.77763092833348]
  Intelligent agents powered by large language models (LLMs) have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines. While their capabilities are promising, these agents also introduce novel vulnerabilities that demand careful consideration for safety. This paper conducts a thorough examination of vulnerabilities in LLM-based agents within scientific domains, shedding light on potential risks associated with their misuse and emphasizing the need for safety measures.
  arXiv  Detail & Related papers  (2024-02-06T18:54:07Z)
• Contrastive Learning of Temporal Distinctiveness for Survival Analysis in Electronic Health Records [10.192973297290136]
  We propose a novel Ontology-aware Temporality-based Contrastive Survival (OTCSurv) analysis framework. OTCSurv uses survival durations from both censored and observed data to define temporal distinctiveness. We conduct experiments using a large EHR dataset to forecast the risk of hospitalized patients who are in danger of developing acute kidney injury (AKI)
  arXiv  Detail & Related papers  (2023-08-24T22:36:22Z)
• Just-In-Time Learning for Operational Risk Assessment in Power Grids [12.939739997360016]
  In a grid with a significant share of renewable generation, operators will need additional tools to evaluate the operational risk. This paper proposes a Just-In-Time Risk Assessment Learning Framework (JITRALF) as an alternative. JITRALF trains risk surrogates, one for each hour in the day, using Machine Learning (ML) to predict the quantities needed to estimate risk.
  arXiv  Detail & Related papers  (2022-09-26T15:11:27Z)
• A Survey of Risk-Aware Multi-Armed Bandits [84.67376599822569]
  We review various risk measures of interest, and comment on their properties. We consider algorithms for the regret minimization setting, where the exploration-exploitation trade-off manifests. We conclude by commenting on persisting challenges and fertile areas for future research.
  arXiv  Detail & Related papers  (2022-05-12T02:20:34Z)
• Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning [80.66108902283388]
  Multi-task learning methods should be used with caution for safety-critical applications, such as clinical risk prediction. Existing asymmetric multi-task learning methods tackle this negative transfer problem by performing knowledge transfer from tasks with low loss to tasks with high loss. We propose a novel temporal asymmetric multi-task learning model that performs knowledge transfer from certain tasks/timesteps to relevant uncertain tasks, based on feature-level uncertainty.
  arXiv  Detail & Related papers  (2020-06-23T06:01:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.