Taxonomy-Driven Analysis of Open-Source AI Risk Mitigation Tools

• URL: http://arxiv.org/abs/2608.07446v1
• Date: Fri, 07 Aug 2026 17:33:09 GMT
• Title: Taxonomy-Driven Analysis of Open-Source AI Risk Mitigation Tools
• Abstract summary: This paper proposes a structured protocol to automate AI risk mitigation through a taxonomy-driven analysis of open-source tools.<n>We map the capabilities of 21 prominent open-source tools to the 32 subcategories of the extended MIT AI Risk Mitigation and Response taxonomy.<n>The analysis reveals a highly skewed landscape in which tools cluster around technical and operational controls, while governance, legal and regulatory, and financial and market controls remain largely unaddressed.
• Score: 0.04510277221887893
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Rapid adoption of large language models (LLMs) in enterprise settings has introduced operational, security, and governance risks. As generative AI applications move from pilot to production, manual harm identification and mitigation are becoming difficult to scale. Although many tools support model evaluation, adversarial testing, runtime guardrails, and observability, the tooling landscape remains fragmented. Tools are typically designed for specific engineering tasks and described in technical terms that do not align with governance frameworks or risk taxonomies, making it difficult to determine which tools address which risks and where critical gaps remain. This paper proposes a structured protocol to automate AI risk mitigation through a taxonomy-driven analysis of open-source LLM evaluation and security tools. We map the capabilities of 21 prominent open-source tools to the 32 subcategories of the extended MIT AI Risk Mitigation and Response Taxonomy. An LLM-assisted retrieval-augmented generation pipeline analyzes source code and documentation to extract capabilities for each taxonomy category. Reliability assessment yielded moderate agreement (Fleiss' Kappa = 0.509) among three independent reviewers. The analysis reveals a highly skewed landscape in which tools cluster around technical and operational controls, while governance, legal and regulatory, and financial and market controls remain largely unaddressed. This motivates a layered risk-mitigation architecture combining tool-based controls with organizational and regulatory processes. The mapping protocol achieved an F1 score of 75.5% after majority voting. Overall, the study provides a practical mapping between enterprise AI risk categories and open-source mitigation capabilities, identifies where human oversight remains necessary, and presents a taxonomy-driven framework applicable to open-source and proprietary solutions.

Related papers

• Governing Bring Your Own AI: A Parameterized Maturity Model [0.0]
  Bring Your Own AI (BYOAI) is a distinct form of Shadow AI in which employee-authenticated personal accounts are used outside of enterprise identity and security controls.<n>Existing frameworks were designed for AI tools managed by organizations, and their coverage does not extend to unmanaged AI tools used with a personal account.<n>We develop a governance model through a systematic review of the literature that produces a risk taxonomy and a framework-engagement profile.
  arXiv  Detail & Related papers  (2026-09-04T15:03:49Z)
• The Eticas AI Risk Taxonomy: Open Infrastructure for Operationalizing AI Audits [2.869158116077567]
  We present the operationalization layer Eticas has built and run, shown end to end on a single risk (PII leakage) against a public benchmark.<n>On GPT-4-0314, a disclosure risk that seven external frameworks require be controlled is measured at 0%, 51%, and 84% disclosure as adversarial conditioning increases.<n>The contribution is the demonstrated bridge from concept to graded finding, anchored by a clean separation of risks from the mechanisms that they surface.
  arXiv  Detail & Related papers  (2026-07-02T14:08:34Z)
• Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governance [0.0]
  This paper contributes a seven-dimension profile distinguishing pure software-engineering, hybrid, and AI-native teams.<n>Because the object of study is framework adequacy rather than human behavior, the evaluation yields derived rather than observed coverage claims.
  arXiv  Detail & Related papers  (2026-07-01T19:31:02Z)
• OmniVL-Guard Pro: A Tool-Augmented Agent for Omnibus Vision-Language Forensics [63.13200245209719]
  We propose textbf OmniVL-Guard Pro, a tool-augmented agent that extends unified forensics from closed-world prediction to open-world clues-driven reasoning.<n>To generate high-quality tool-reasoning trajectories, we introduce textbfTree-Structured Self-Evolving Tool Trajectory Generation.<n>We also propose textbfChecker-Guided Agentic Reinforcement Learning, which provides process-level supervision to penalize cases where the answer is correct but the reasoning is distorted.
  arXiv  Detail & Related papers  (2026-05-16T12:26:04Z)
• AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration [49.050688065974605]
  Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models.<n>We propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution.
  arXiv  Detail & Related papers  (2026-04-01T15:17:45Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report v1.5 [61.787178868669265]
  This technical report presents an updated and granular assessment of five critical dimensions: cyber offense, persuasion and manipulation, strategic deception, uncontrolled AI R&D, and self-replication.<n>This work reflects our current understanding of AI frontier risks and urges collective action to mitigate these challenges.
  arXiv  Detail & Related papers  (2026-02-16T04:30:06Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• Mapping AI Risk Mitigations: Evidence Scan and Preliminary AI Risk Mitigation Taxonomy [35.22340964134219]
  The landscape of AI risk mitigation frameworks is fragmented, uses inconsistent terminology, and has gaps in coverage.<n>This paper introduces a preliminary AI Risk Mitigation Taxonomy to organize AI risk mitigations and provide a common frame of reference.<n>The taxonomy was developed through a rapid evidence scan of 13 AI risk mitigation frameworks published between 2023-2025, which were extracted into a living database of 831 AI risk mitigations.
  arXiv  Detail & Related papers  (2025-12-12T03:26:29Z)
• AI Bill of Materials and Beyond: Systematizing Security Assurance through the AI Risk Scanning (AIRS) Framework [31.261980405052938]
  Assurance for artificial intelligence (AI) systems remains fragmented across software supply-chain security, adversarial machine learning, and governance documentation.<n>This paper introduces the AI Risk Scanning (AIRS) Framework, a threat-model-based, evidence-generating framework designed to operationalize AI assurance.
  arXiv  Detail & Related papers  (2025-11-16T16:10:38Z)
• Mapping LLM Security Landscapes: A Comprehensive Stakeholder Risk Assessment Proposal [0.0]
  We propose a risk assessment process using tools like the risk rating methodology which is used for traditional systems. We conduct scenario analysis to identify potential threat agents and map the dependent system components against vulnerability factors. We also map threats against three key stakeholder groups.
  arXiv  Detail & Related papers  (2024-03-20T05:17:22Z)
• RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models [46.476439550746136]
  Large language model (LLM) applications in cloud root cause analysis (RCA) have been actively explored recently. We present RCAgent, a tool-augmented LLM autonomous agent framework for practical and privacy-aware industrial RCA usage. Running on an internally deployed model rather than GPT families, RCAgent is capable of free-form data collection and comprehensive analysis with tools.
  arXiv  Detail & Related papers  (2023-10-25T03:53:31Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.