When Openness Fails: Lessons from System Safety for Assessing Openness in AI

• URL: http://arxiv.org/abs/2510.10732v1
• Date: Sun, 12 Oct 2025 18:08:00 GMT
• Title: When Openness Fails: Lessons from System Safety for Assessing Openness in AI
• Abstract summary: We adapt five lessons from system safety that offer guidance on how openness can be evaluated at the system level.<n>To evaluate whether the intended effects of openness - such as democratization and autonomy - are realized, we need a more holistic approach.
• Score: 2.2077181316192473
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most frameworks for assessing the openness of AI systems use narrow criteria such as availability of data, model, code, documentation, and licensing terms. However, to evaluate whether the intended effects of openness - such as democratization and autonomy - are realized, we need a more holistic approach that considers the context of release: who will reuse the system, for what purposes, and under what conditions. To this end, we adapt five lessons from system safety that offer guidance on how openness can be evaluated at the system level.

Related papers

• Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies [57.521647436515785]
  We define frontier AI auditing as rigorous third-party verification of frontier AI developers' safety and security claims.<n>We introduce AI Assurance Levels (AAL-1 to AAL-4), ranging from time-bounded system audits to continuous, deception-resilient verification.
  arXiv  Detail & Related papers  (2026-01-16T18:44:09Z)
• Variance-Bounded Evaluation of Entity-Centric AI Systems Without Ground Truth: Theory and Measurement [0.0]
  We introduce VB-Score, a variance-bounded evaluation framework for entity-centric AI systems.<n> VB-Score enumerates plausible interpretations through constraint relaxation and Monte Carlo sampling.<n>It then evaluates system outputs by their expected success across interpretations, penalized by variance to assess robustness of the system.
  arXiv  Detail & Related papers  (2025-09-26T07:54:38Z)
• Toward a Public and Secure Generative AI: A Comparative Analysis of Open and Closed LLMs [0.0]
  This study aims to critically evaluate and compare the characteristics, opportunities, and challenges of open and closed generative AI models.<n>The proposed framework outlines key dimensions, openness, public governance, and security, as essential pillars for shaping the future of trustworthy and inclusive Gen AI.
  arXiv  Detail & Related papers  (2025-05-15T15:21:09Z)
• Re-evaluating Open-ended Evaluation of Large Language Models [50.23008729038318]
  We show that the current Elo-based rating systems can be susceptible to and even reinforce biases in data, intentional or accidental.<n>We propose evaluation as a 3-player game, and introduce novel game-theoretic solution concepts to ensure robustness to redundancy.
  arXiv  Detail & Related papers  (2025-02-27T15:07:47Z)
• Toward a Principled Framework for Disclosure Avoidance [34.13680274689903]
  As data users' needs change, agencies must redesign the disclosure avoidance system(s) they use.<n>System's ability to calibrate the strength of protection to suit the underlying disclosure risk of the data is a worthwhile feature.<n>This paper proposes a framework for distinguishing these inherent features from the implementation decisions that need to be made independent of the system selected.
  arXiv  Detail & Related papers  (2025-02-10T22:58:06Z)
• Pessimistic Evaluation [58.736490198613154]
  We argue that evaluating information access systems assumes utilitarian values not aligned with traditions of information access based on equal access. We advocate for pessimistic evaluation of information access systems focusing on worst case utility.
  arXiv  Detail & Related papers  (2024-10-17T15:40:09Z)
• Trustworthiness in Retrieval-Augmented Generation Systems: A Survey [59.26328612791924]
  Retrieval-Augmented Generation (RAG) has quickly grown into a pivotal paradigm in the development of Large Language Models (LLMs) We propose a unified framework that assesses the trustworthiness of RAG systems across six key dimensions: factuality, robustness, fairness, transparency, accountability, and privacy.
  arXiv  Detail & Related papers  (2024-09-16T09:06:44Z)
• The Gradient of Generative AI Release: Methods and Considerations [0.0]
  We propose a framework to assess six levels of access to generative AI systems. We show trends in generative system release over time, noting closedness among large companies for powerful systems. We also enumerate safety controls and guardrails for generative systems and necessary investments to improve future releases.
  arXiv  Detail & Related papers  (2023-02-05T23:05:11Z)
• How to Evaluate Explainability? -- A Case for Three Criteria [0.0]
  We will provide a multidisciplinary motivation for three quality criteria concerning the information that systems should provide. Our aim is to fuel the discussion regarding these criteria, such as adequate evaluation methods for them will be conceived.
  arXiv  Detail & Related papers  (2022-09-01T11:22:50Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)
• Towards an Interface Description Template for AI-enabled Systems [77.34726150561087]
  Reuse is a common system architecture approach that seeks to instantiate a system architecture with existing components. There is currently no framework that guides the selection of necessary information to assess their portability to operate in a system different than the one for which the component was originally purposed. We present ongoing work on establishing an interface description template that captures the main information of an AI-enabled component.
  arXiv  Detail & Related papers  (2020-07-13T20:30:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.