RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting

• URL: http://arxiv.org/abs/2504.08952v1
• Date: Fri, 11 Apr 2025 20:10:15 GMT
• Title: RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting
• Abstract summary: Only 14% of model cards mention risks, out of which 96% copying content from a small set of cards, leading to a lack of actionable insights.<n>RiskRAG is a Retrieval Augmented Generation based risk reporting solution guided by five design requirements.<n>Drawing from 450K model cards and 600 real-world incidents, RiskRAG pre-populates contextualized risk reports.
• Score: 4.375071270418409
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Risk reporting is essential for documenting AI models, yet only 14% of model cards mention risks, out of which 96% copying content from a small set of cards, leading to a lack of actionable insights. Existing proposals for improving model cards do not resolve these issues. To address this, we introduce RiskRAG, a Retrieval Augmented Generation based risk reporting solution guided by five design requirements we identified from literature, and co-design with 16 developers: identifying diverse model-specific risks, clearly presenting and prioritizing them, contextualizing for real-world uses, and offering actionable mitigation strategies. Drawing from 450K model cards and 600 real-world incidents, RiskRAG pre-populates contextualized risk reports. A preliminary study with 50 developers showed that they preferred RiskRAG over standard model cards, as it better met all the design requirements. A final study with 38 developers, 40 designers, and 37 media professionals showed that RiskRAG improved their way of selecting the AI model for a specific application, encouraging a more careful and deliberative decision-making. The RiskRAG project page is accessible at: https://social-dynamics.net/ai-risks/card.

Related papers

• Emergent Strategic Reasoning Risks in AI: A Taxonomy-Driven Evaluation Framework [63.74295981594549]
  Large language models (LLMs) engage in behaviors that serve their own objectives.<n>These include, but are not limited to, deception (intentionally misleading users or evaluators), evaluation gaming (strategically manipulating performance during safety testing), and reward hacking.<n>We introduce ESRRSim, a taxonomy-driven agentic framework for automated behavioral risk evaluation.
  arXiv  Detail & Related papers  (2026-04-23T23:44:01Z)
• Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report v1.5 [61.787178868669265]
  This technical report presents an updated and granular assessment of five critical dimensions: cyber offense, persuasion and manipulation, strategic deception, uncontrolled AI R&D, and self-replication.<n>This work reflects our current understanding of AI frontier risks and urges collective action to mitigate these challenges.
  arXiv  Detail & Related papers  (2026-02-16T04:30:06Z)
• Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse [50.87630846876635]
  We develop nine detailed cyber risk models.<n>Each model decomposes attacks into steps using the MITRE ATT&CK framework.<n>Individual estimates are aggregated through Monte Carlo simulation.
  arXiv  Detail & Related papers  (2025-12-09T17:54:17Z)
• The Role of Risk Modeling in Advanced AI Risk Management [33.357295564462284]
  Rapidly advancing artificial intelligence (AI) systems introduce novel, uncertain, and potentially catastrophic risks.<n>Managing these risks requires a mature risk-management infrastructure whose cornerstone is rigorous risk modeling.<n>We argue that advanced-AI governance should adopt a similar dual approach and that verifiable, provably-safe AI architectures are urgently needed.
  arXiv  Detail & Related papers  (2025-12-09T15:37:33Z)
• The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms [7.37865739449202]
  We analyzed nearly 460,000 AI model cards from Hugging Face to examine how developers report risks.<n>We compared these with risks identified by researchers in the MIT Risk Repository and with real-world incidents from the AI Incident Database.<n>Our findings show the need for clearer, structured risk reporting that helps developers think about human-interaction and systemic risks early in the design process.
  arXiv  Detail & Related papers  (2025-08-21T07:07:41Z)
• Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report [51.17413460785022]
  This report presents a comprehensive assessment of their frontier risks.<n>We identify critical risks in seven areas: cyber offense, biological and chemical risks, persuasion and manipulation, uncontrolled autonomous AI R&D, strategic deception and scheming, self-replication, and collusion.
  arXiv  Detail & Related papers  (2025-07-22T12:44:38Z)
• Developing a Risk Identification Framework for Foundation Model Uses [7.013133148085937]
  There is little guidance for practitioners on how to determine which risks are relevant for a given foundation model use.<n>We identify challenges for building a foundation model risk identification framework and adapt ideas from usage governance to synthesize four design requirements.
  arXiv  Detail & Related papers  (2025-06-01T23:37:41Z)
• No Query, No Access [50.18709429731724]
  We introduce the textbfVictim Data-based Adrial Attack (VDBA), which operates using only victim texts.<n>To prevent access to the victim model, we create a shadow dataset with publicly available pre-trained models and clustering methods.<n>Experiments on the Emotion and SST5 datasets show that VDBA outperforms state-of-the-art methods, achieving an ASR improvement of 52.08%.
  arXiv  Detail & Related papers  (2025-05-12T06:19:59Z)
• Teaching Models to Understand (but not Generate) High-risk Data [47.384144793060955]
  We introduce Selective Loss to Understand but Not Generate (SLUNG)<n>SLUNG is a pre-training paradigm through which models learn to understand high-risk data without learning to generate it.<n>We show that SLUNG consistently improves models' understanding of high-risk data without increasing its generation.
  arXiv  Detail & Related papers  (2025-05-05T22:24:06Z)
• Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
  We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications. Our research identifies two critical latent factors affecting RAG's confidence in its predictions. We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
  arXiv  Detail & Related papers  (2024-09-24T14:52:14Z)
• Risks and NLP Design: A Case Study on Procedural Document QA [52.557503571760215]
  We argue that clearer assessments of risks and harms to users will be possible when we specialize the analysis to more concrete applications and their plausible users. We conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
  arXiv  Detail & Related papers  (2024-08-16T17:23:43Z)
• EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents [53.717918131568936]
  Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. Foundation models as the "brain" of EAI agents for high-level task planning have shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios.
  arXiv  Detail & Related papers  (2024-08-08T13:19:37Z)
• "Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models [74.05368440735468]
  Retrieval-Augmented Generative (RAG) models enhance Large Language Models (LLMs) In this paper, we demonstrate a security threat where adversaries can exploit the openness of these knowledge bases.
  arXiv  Detail & Related papers  (2024-06-26T05:36:23Z)
• CRiskEval: A Chinese Multi-Level Risk Evaluation Benchmark Dataset for Large Language Models [46.93425758722059]
  CRiskEval is a Chinese dataset meticulously designed for gauging the risk proclivities inherent in large language models (LLMs) We define a new risk taxonomy with 7 types of frontier risks and 4 safety levels, including extremely hazardous,moderately hazardous, neutral and safe. The dataset consists of 14,888 questions that simulate scenarios related to predefined 7 types of frontier risks.
  arXiv  Detail & Related papers  (2024-06-07T08:52:24Z)
• C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models [57.10361282229501]
  We propose C-RAG, the first framework to certify generation risks for RAG models. Specifically, we provide conformal risk analysis for RAG models and certify an upper confidence bound of generation risks. We prove that RAG achieves a lower conformal generation risk than that of a single LLM when the quality of the retrieval model and transformer is non-trivial.
  arXiv  Detail & Related papers  (2024-02-05T16:46:16Z)
• Modeling Risk in Reinforcement Learning: A Literature Mapping [0.0]
  We perform a systematic literature mapping with the objective to characterize risk in safe RL. Based on the obtained results, we present definitions, characteristics, and types of risk that hold on multiple application domains.
  arXiv  Detail & Related papers  (2023-12-08T18:26:08Z)
• AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosures [0.8702432681310399]
  We propose a risk profiling standard which can guide downstream decision-making. The standard is built on our proposed taxonomy of AI risks, which reflects a high-level categorization of the wide variety of risks proposed in the literature. We apply this methodology to a number of prominent AI systems using publicly available information.
  arXiv  Detail & Related papers  (2023-09-22T20:45:15Z)
• Typology of Risks of Generative Text-to-Image Models [1.933681537640272]
  This paper investigates the direct risks and harms associated with modern text-to-image generative models, such as DALL-E and Midjourney. Our review reveals significant knowledge gaps concerning the understanding and treatment of these risks despite some already being addressed. We identify 22 distinct risk types, spanning issues from data bias to malicious use.
  arXiv  Detail & Related papers  (2023-07-08T20:33:30Z)
• Assessing Language Model Deployment with Risk Cards [30.927131991814637]
  RiskCards are a framework for structured assessment and documentation of risks associated with language models. Each RiskCard makes clear the routes for the risk to manifest harm, their placement in harm, and example prompt-output pairs.
  arXiv  Detail & Related papers  (2023-03-31T16:45:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.