Assessing Language Model Deployment with Risk Cards

• URL: http://arxiv.org/abs/2303.18190v1
• Date: Fri, 31 Mar 2023 16:45:42 GMT
• Title: Assessing Language Model Deployment with Risk Cards
• Abstract summary: RiskCards are a framework for structured assessment and documentation of risks associated with language models. Each RiskCard makes clear the routes for the risk to manifest harm, their placement in harm, and example prompt-output pairs.
• Score: 30.927131991814637
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces RiskCards, a framework for structured assessment and documentation of risks associated with an application of language models. As with all language, text generated by language models can be harmful, or used to bring about harm. Automating language generation adds both an element of scale and also more subtle or emergent undesirable tendencies to the generated text. Prior work establishes a wide variety of language model harms to many different actors: existing taxonomies identify categories of harms posed by language models; benchmarks establish automated tests of these harms; and documentation standards for models, tasks and datasets encourage transparent reporting. However, there is no risk-centric framework for documenting the complexity of a landscape in which some risks are shared across models and contexts, while others are specific, and where certain conditions may be required for risks to manifest as harms. RiskCards address this methodological gap by providing a generic framework for assessing the use of a given language model in a given scenario. Each RiskCard makes clear the routes for the risk to manifest harm, their placement in harm taxonomies, and example prompt-output pairs. While RiskCards are designed to be open-source, dynamic and participatory, we present a "starter set" of RiskCards taken from a broad literature survey, each of which details a concrete risk presentation. Language model RiskCards initiate a community knowledge base which permits the mapping of risks and harms to a specific model or its application scenario, ultimately contributing to a better, safer and shared understanding of the risk landscape.

Related papers

• SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs [56.34354368047577]
  Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts.<n>We introduce SurakshaEval, a novel safety benchmark composed of human-written prompts spanning real-world scenarios.<n>We benchmark a broad range of state-of-the-art LLMs on SurakshaEval, establish baseline safety performance, and identify recurring failure modes.
  arXiv  Detail & Related papers  (2026-08-08T02:03:33Z)
• Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models [54.10540442330978]
  Existing benchmarks are typically multilingual but text-only, or multimodal but monolingual.<n>Recent multilingual red-teaming efforts render harmful prompts into images, yet rely heavily on typography-style visuals.<n>We introduce a benchmark of 100,440 harmful image-text pairs across 10 languages, explicitly partitioned into image-dominant and text-dominant subsets.
  arXiv  Detail & Related papers  (2026-01-30T09:18:13Z)
• RAG Security and Privacy: Formalizing the Threat Model and Attack Surface [4.823988025629304]
  Retrieval-Augmented Generation (RAG) is an emerging approach in natural language processing that combines large language models (LLMs) with external document retrieval to produce more accurate and grounded responses.<n>Existing research has demonstrated that RAGs can leak sensitive information through training data memorization or adversarial prompts, and RAG systems inherit many of these vulnerabilities.<n>Despite these risks, there is currently no formal framework that defines the threat landscape for RAG systems.
  arXiv  Detail & Related papers  (2025-09-24T17:11:35Z)
• Exploring the Secondary Risks of Large Language Models [17.845215420030467]
  We introduce secondary risks marked by harmful or misleading behaviors during benign prompts.<n>Unlike adversarial attacks, these risks stem from imperfect generalization and often evade standard safety mechanisms.<n>We propose SecLens, a black-box, multi-objective search framework that efficiently elicits secondary risk behaviors.
  arXiv  Detail & Related papers  (2025-06-14T07:31:52Z)
• Developing a Risk Identification Framework for Foundation Model Uses [7.013133148085937]
  There is little guidance for practitioners on how to determine which risks are relevant for a given foundation model use.<n>We identify challenges for building a foundation model risk identification framework and adapt ideas from usage governance to synthesize four design requirements.
  arXiv  Detail & Related papers  (2025-06-01T23:37:41Z)
• RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting [4.375071270418409]
  Only 14% of model cards mention risks, out of which 96% copying content from a small set of cards, leading to a lack of actionable insights. RiskRAG is a Retrieval Augmented Generation based risk reporting solution guided by five design requirements. Drawing from 450K model cards and 600 real-world incidents, RiskRAG pre-populates contextualized risk reports.
  arXiv  Detail & Related papers  (2025-04-11T20:10:15Z)
• DODGE: Ontology-Aware Risk Assessment via Object-Oriented Disruption Graphs [0.9387233631570749]
  The Common Ontology of Value and Risk (COVER) highlights how the role of objects and their relationships remains pivotal to performing transparent, complete and accountable risk assessment. We operationalize some of the notions proposed by COVER by presenting a new framework for risk assessment: DODGE.
  arXiv  Detail & Related papers  (2024-12-18T15:44:04Z)
• On the Privacy Risk of In-context Learning [36.633860818454984]
  We show that deploying prompted models presents a significant privacy risk for the data used within the prompt. We also observe that the privacy risk of prompted models exceeds fine-tuned models at the same utility levels.
  arXiv  Detail & Related papers  (2024-11-15T17:11:42Z)
• Risks and NLP Design: A Case Study on Procedural Document QA [52.557503571760215]
  We argue that clearer assessments of risks and harms to users will be possible when we specialize the analysis to more concrete applications and their plausible users. We conduct a risk-oriented error analysis that could then inform the design of a future system to be deployed with lower risk of harm and better performance.
  arXiv  Detail & Related papers  (2024-08-16T17:23:43Z)
• Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference [3.422309388045878]
  Large language models (LLMs) such as ChatGPT are known to pose important risks. misplaced confidence arises from over-confidence or under-confidence, that the models have in their inference. We propose an experimental framework consisting of a two-level inference architecture and appropriate metrics for measuring such risks.
  arXiv  Detail & Related papers  (2024-08-04T05:24:32Z)
• Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights [50.89022445197919]
  We propose a speech-specific risk taxonomy, covering 8 risk categories under hostility (malicious sarcasm and threats), malicious imitation (age, gender, ethnicity), and stereotypical biases (age, gender, ethnicity) Based on the taxonomy, we create a small-scale dataset for evaluating current LMMs capability in detecting these categories of risk.
  arXiv  Detail & Related papers  (2024-06-25T10:08:45Z)
• C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models [57.10361282229501]
  We propose C-RAG, the first framework to certify generation risks for RAG models. Specifically, we provide conformal risk analysis for RAG models and certify an upper confidence bound of generation risks. We prove that RAG achieves a lower conformal generation risk than that of a single LLM when the quality of the retrieval model and transformer is non-trivial.
  arXiv  Detail & Related papers  (2024-02-05T16:46:16Z)
• Multilingual Jailbreak Challenges in Large Language Models [96.74878032417054]
  In this study, we reveal the presence of multilingual jailbreak challenges within large language models (LLMs) We consider two potential risky scenarios: unintentional and intentional. We propose a novel textscSelf-Defense framework that automatically generates multilingual training data for safety fine-tuning.
  arXiv  Detail & Related papers  (2023-10-10T09:44:06Z)
• Typology of Risks of Generative Text-to-Image Models [1.933681537640272]
  This paper investigates the direct risks and harms associated with modern text-to-image generative models, such as DALL-E and Midjourney. Our review reveals significant knowledge gaps concerning the understanding and treatment of these risks despite some already being addressed. We identify 22 distinct risk types, spanning issues from data bias to malicious use.
  arXiv  Detail & Related papers  (2023-07-08T20:33:30Z)
• Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey [50.58063811745676]
  This work provides a survey of practical methods for addressing potential threats and societal harms from language generation models. We draw on several prior works' of language model risks to present a structured overview of strategies for detecting and ameliorating different kinds of risks/harms of language generators.
  arXiv  Detail & Related papers  (2022-10-14T10:43:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.