AI Safety Frameworks Should Include Procedures for Model Access Decisions

• URL: http://arxiv.org/abs/2411.10547v2
• Date: Sun, 01 Dec 2024 13:41:44 GMT
• Title: AI Safety Frameworks Should Include Procedures for Model Access Decisions
• Abstract summary: The downstream use cases, benefits, and risks of AI models depend significantly on what sort of access is provided to the model.<n>Existing safety frameworks and AI developer usage policies recognise that the risk posed by a given model depends on the level of access provided to a given audience.<n>This paper proposes that frontier AI companies build on existing safety frameworks by outlining transparent procedures for making decisions about model access.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The downstream use cases, benefits, and risks of AI models depend significantly on what sort of access is provided to the model, and who it is provided to. Though existing safety frameworks and AI developer usage policies recognise that the risk posed by a given model depends on the level of access provided to a given audience, the procedures they use to make decisions about model access are ad hoc, opaque, and lacking in empirical substantiation. This paper consequently proposes that frontier AI companies build on existing safety frameworks by outlining transparent procedures for making decisions about model access, which we term Responsible Access Policies (RAPs). We recommend that, at a minimum, RAPs should include the following: i) processes for empirically evaluating model capabilities given different styles of access, ii) processes for assessing the risk profiles of different categories of user, and iii) clear and robust pre-commitments regarding when to grant or revoke specific types of access for particular groups under specified conditions.

Related papers

• IP Protection in the Era of Visual Generative AI: A Survey [70.06761647752164]
  This survey presents a two-dimensional taxonomy for IP protection in visual generative models.<n>The primary axis is a Control Logic View, which classifies methods into Information Exposure Control, Generative Behavior Constraint, and Attribution & Accountability.<n>The secondary axis distinguishes Data IP from Model IP as cross-cutting asset dimensions.
  arXiv  Detail & Related papers  (2026-08-13T00:26:41Z)
• Harmonizing AI Safety Thresholds [0.0]
  We develop a methodology for deriving harmonized thresholds across three risk domains.<n>For misuse risks, we take expected harm as the key primitive and use an explicit risk-modeling approach.<n>For automated AI R&D, we base our proposed threshold on the observed rate of AI progress rather than expected harm.
  arXiv  Detail & Related papers  (2026-07-17T16:45:30Z)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation [68.53387633351484]
  We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions.<n> CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step.<n>To rigorously evaluate this paradigm, we introduce Phone-Harm, a new benchmark of mobile safety violations with step-level harm labels.
  arXiv  Detail & Related papers  (2026-04-10T09:41:21Z)
• Beyond the Binary: A nuanced path for open-weight advanced AI [0.0]
  Open-weight advanced AI models are systems whose parameters are freely available for download and adaptation.<n>This report moves beyond the binary framing of open'' versus closed'' AI.<n>We find significant disparities in safety practices across developers and jurisdictions.<n>We propose a tiered, safety-anchored approach to model release, where openness is determined by rigorous risk assessment and demonstrated safety.
  arXiv  Detail & Related papers  (2026-02-23T10:26:23Z)
• Expanding External Access To Frontier AI Models For Dangerous Capability Evaluations [41.72971769197687]
  The EU General-Purpose AI Code of Practice calls for "appropriate access", but does not specify what this means in practice.<n>There is no common framework for describing different types and levels of evaluator access.<n>We propose a taxonomy of access methods for dangerous capability evaluations.
  arXiv  Detail & Related papers  (2026-01-17T05:33:04Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• An Ontology-Based Approach to Security Risk Identification of Container Deployments in OT Contexts [1.826848871278733]
  Security risk identification for OT container deployments is challenged by hybrid IT/OT architectures, fragmented stakeholder knowledge, and continuous system changes.<n>We propose a model-based approach, implemented as the Container Security Risk Ontology (CSRO)<n>CSRO integrates five key domains: adversarial behaviour, contextual assumptions, attack scenarios, risk assessment rules, and container security artefacts.
  arXiv  Detail & Related papers  (2026-01-07T15:20:19Z)
• A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives [65.3369988566853]
  Recent studies have demonstrated that adversaries can replicate a target model's functionality.<n>Model Extraction Attacks pose threats to intellectual property, privacy, and system security.<n>We propose a novel taxonomy that classifies MEAs according to attack mechanisms, defense approaches, and computing environments.
  arXiv  Detail & Related papers  (2025-08-20T19:49:59Z)
• Systematic Hazard Analysis for Frontier AI using STPA [0.0]
  frontier AI companies currently do not describe in detail any structured approach to identifying and analysing hazards.<n>A (Systems-Theoretic Process Analysis) is a systematic methodology for identifying how complex systems can become unsafe, leading to hazards.<n>We evaluateA's ability to broaden the scope, improve traceability and strengthen the robustness of safety assurance for frontier AI systems.
  arXiv  Detail & Related papers  (2025-06-02T15:28:34Z)
• No Query, No Access [50.18709429731724]
  We introduce the textbfVictim Data-based Adrial Attack (VDBA), which operates using only victim texts.<n>To prevent access to the victim model, we create a shadow dataset with publicly available pre-trained models and clustering methods.<n>Experiments on the Emotion and SST5 datasets show that VDBA outperforms state-of-the-art methods, achieving an ASR improvement of 52.08%.
  arXiv  Detail & Related papers  (2025-05-12T06:19:59Z)
• SPoRt -- Safe Policy Ratio: Certified Training and Deployment of Task Policies in Model-Free RL [54.022106606140774]
  We present theoretical results that provide a bound on the probability of violating a safety property for a new task-specific policy in a model-free, episodic setup. We also present SPoRt, which enables the user to trade off safety guarantees in exchange for task-specific performance.
  arXiv  Detail & Related papers  (2025-04-08T19:09:07Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability. The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• DocMIA: Document-Level Membership Inference Attacks against DocVQA Models [52.13818827581981]
  We introduce two novel membership inference attacks tailored specifically to DocVQA models. Our methods outperform existing state-of-the-art membership inference attacks across a variety of DocVQA models and datasets.
  arXiv  Detail & Related papers  (2025-02-06T00:58:21Z)
• Safety case template for frontier AI: A cyber inability argument [2.2628353000034065]
  We propose a safety case template for offensive cyber capabilities. We identify a number of risk models, derive proxy tasks from the risk models, define evaluation settings for the proxy tasks, and connect those with evaluation results.
  arXiv  Detail & Related papers  (2024-11-12T18:45:08Z)
• Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks [65.21536453075275]
  We focus on the summarization task and investigate the membership inference (MI) attack. We exploit text similarity and the model's resistance to document modifications as potential MI signals. We discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2023-10-20T05:44:39Z)
• AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosures [0.8702432681310399]
  We propose a risk profiling standard which can guide downstream decision-making. The standard is built on our proposed taxonomy of AI risks, which reflects a high-level categorization of the wide variety of risks proposed in the literature. We apply this methodology to a number of prominent AI systems using publicly available information.
  arXiv  Detail & Related papers  (2023-09-22T20:45:15Z)
• REX: Rapid Exploration and eXploitation for AI Agents [103.68453326880456]
  We propose an enhanced approach for Rapid Exploration and eXploitation for AI Agents called REX. REX introduces an additional layer of rewards and integrates concepts similar to Upper Confidence Bound (UCB) scores, leading to more robust and efficient AI agent performance.
  arXiv  Detail & Related papers  (2023-07-18T04:26:33Z)
• Online Safety Property Collection and Refinement for Safe Deep Reinforcement Learning in Mapless Navigation [79.89605349842569]
  We introduce the Collection and Refinement of Online Properties (CROP) framework to design properties at training time. CROP employs a cost signal to identify unsafe interactions and use them to shape safety properties. We evaluate our approach in several robotic mapless navigation tasks and demonstrate that the violation metric computed with CROP allows higher returns and lower violations over previous Safe DRL approaches.
  arXiv  Detail & Related papers  (2023-02-13T21:19:36Z)
• Risk Consistent Multi-Class Learning from Label Proportions [64.0125322353281]
  This study addresses a multiclass learning from label proportions (MCLLP) setting in which training instances are provided in bags. Most existing MCLLP methods impose bag-wise constraints on the prediction of instances or assign them pseudo-labels. A risk-consistent method is proposed for instance classification using the empirical risk minimization framework.
  arXiv  Detail & Related papers  (2022-03-24T03:49:04Z)
• Risk-Aware Fine-Grained Access Control in Cyber-Physical Contexts [12.138525287184061]
  RASA is a context-sensitive access authorization approach and mechanism leveraging unsupervised machine learning to automatically infer risk-based authorization decision boundaries. We explore RASA in a healthcare usage environment, wherein cyber and physical conditions create context-specific risks for protecting private health information.
  arXiv  Detail & Related papers  (2021-08-29T03:38:45Z)
• SMT-Based Safety Verification of Data-Aware Processes under Ontologies (Extended Version) [71.12474112166767]
  We introduce a variant of one of the most investigated models in this spectrum, namely simple artifact systems (SASs) This DL, enjoying suitable model-theoretic properties, allows us to define SASs to which backward reachability can still be applied, leading to decidability in PSPACE of the corresponding safety problems.
  arXiv  Detail & Related papers  (2021-08-27T15:04:11Z)
• An Abstraction-based Method to Verify Multi-Agent Deep Reinforcement-Learning Behaviours [8.95294551927446]
  Multi-agent reinforcement learning (RL) often struggles to ensure the safe behaviours of the learning agents. We present a methodology that combines formal verification with (deep) RL algorithms to guarantee the satisfaction of formally-specified safety constraints.
  arXiv  Detail & Related papers  (2021-02-02T11:12:30Z)
• Evaluating the Safety of Deep Reinforcement Learning Models using Semi-Formal Verification [81.32981236437395]
  We present a semi-formal verification approach for decision-making tasks based on interval analysis. Our method obtains comparable results over standard benchmarks with respect to formal verifiers. Our approach allows to efficiently evaluate safety properties for decision-making models in practical applications.
  arXiv  Detail & Related papers  (2020-10-19T11:18:06Z)
• An Automatic Attribute Based Access Control Policy Extraction from Access Logs [5.142415132534397]
  An attribute-based access control (ABAC) model provides a more flexible approach for addressing the authorization needs of complex and dynamic systems. We present a methodology for automatically learning ABAC policy rules from access logs of a system to simplify the policy development process.
  arXiv  Detail & Related papers  (2020-03-16T15:08:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.