The Pitfalls of "Security by Obscurity" And What They Mean for Transparent AI

• URL: http://arxiv.org/abs/2501.18669v1
• Date: Thu, 30 Jan 2025 17:04:35 GMT
• Title: The Pitfalls of "Security by Obscurity" And What They Mean for Transparent AI
• Authors: Peter Hall, Olivia Mundahl, Sunoo Park,
• Abstract summary: We identify three key themes in the security community's perspective on the benefits of transparency. We then provide a case study discussion on how transparency has shaped the research subfield of anonymization.
• Score: 4.627627425427264
• License:
• Abstract: Calls for transparency in AI systems are growing in number and urgency from diverse stakeholders ranging from regulators to researchers to users (with a comparative absence of companies developing AI). Notions of transparency for AI abound, each addressing distinct interests and concerns. In computer security, transparency is likewise regarded as a key concept. The security community has for decades pushed back against so-called security by obscurity -- the idea that hiding how a system works protects it from attack -- against significant pressure from industry and other stakeholders. Over the decades, in a community process that is imperfect and ongoing, security researchers and practitioners have gradually built up some norms and practices around how to balance transparency interests with possible negative side effects. This paper asks: What insights can the AI community take from the security community's experience with transparency? We identify three key themes in the security community's perspective on the benefits of transparency and their approach to balancing transparency against countervailing interests. For each, we investigate parallels and insights relevant to transparency in AI. We then provide a case study discussion on how transparency has shaped the research subfield of anonymization. Finally, shifting our focus from similarities to differences, we highlight key transparency issues where modern AI systems present challenges different from other kinds of security-critical systems, raising interesting open questions for the security and AI communities alike.

Related papers

• Transparency, Security, and Workplace Training & Awareness in the Age of Generative AI [0.0]
  As AI technologies advance, ethical considerations, transparency, data privacy, and their impact on human labor intersect with the drive for innovation and efficiency. Our research explores publicly accessible large language models (LLMs) that often operate on the periphery, away from mainstream scrutiny. Specifically, we examine Gab AI, a platform that centers around unrestricted communication and privacy, allowing users to interact freely without censorship.
  arXiv  Detail & Related papers  (2024-12-19T17:40:58Z)
• Enhancing transparency in AI-powered customer engagement [0.0]
  This paper addresses the critical challenge of building consumer trust in AI-powered customer engagement. Despite the potential of AI to revolutionise business operations, widespread concerns about misinformation and the opacity of AI decision-making processes hinder trust. By adopting a holistic approach to transparency and explainability, businesses can cultivate trust in AI technologies.
  arXiv  Detail & Related papers  (2024-09-13T20:26:11Z)
• AI Risk Management Should Incorporate Both Safety and Security [185.68738503122114]
  We argue that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security. We introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security.
  arXiv  Detail & Related papers  (2024-05-29T21:00:47Z)
• Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems [3.9143193313607085]
  We present a reflective analysis of transparency requirements and impacts in AI knowledge systems. We formulate transparency as a key mediator in shaping different ways of seeing. We identify three transparency dimensions necessary to realize the value of AI knowledge systems.
  arXiv  Detail & Related papers  (2024-01-17T18:47:30Z)
• Path To Gain Functional Transparency In Artificial Intelligence With Meaningful Explainability [0.0]
  As AI systems become increasingly sophisticated, ensuring their transparency and explainability becomes crucial. We propose a design for user-centered compliant-by-design transparency in transparent systems. By providing a comprehensive understanding of the challenges associated with transparency in AI systems, we aim to facilitate the development of AI systems that are accountable, trustworthy, and aligned with societal values.
  arXiv  Detail & Related papers  (2023-10-13T04:25:30Z)
• Representation Engineering: A Top-Down Approach to AI Transparency [132.0398250233924]
  We identify and characterize the emerging area of representation engineering (RepE) RepE places population-level representations, rather than neurons or circuits, at the center of analysis. We showcase how these methods can provide traction on a wide range of safety-relevant problems.
  arXiv  Detail & Related papers  (2023-10-02T17:59:07Z)
• Users are the North Star for AI Transparency [111.5679109784322]
  Despite widespread calls for transparent artificial intelligence systems, the term is too overburdened with disparate meanings to express precise policy aims or to orient concrete lines of research. Part of why this happens is that a clear ideal of AI transparency goes unsaid in this body of work. We explicitly name such a north star -- transparency that is user-centered, user-appropriate, and honest.
  arXiv  Detail & Related papers  (2023-03-09T18:53:29Z)
• Designing for Responsible Trust in AI Systems: A Communication Perspective [56.80107647520364]
  We draw from communication theories and literature on trust in technologies to develop a conceptual model called MATCH. We highlight transparency and interaction as AI systems' affordances that present a wide range of trustworthiness cues to users. We propose a checklist of requirements to help technology creators identify appropriate cues to use.
  arXiv  Detail & Related papers  (2022-04-29T00:14:33Z)
• Dimensions of Transparency in NLP Applications [64.16277166331298]
  Broader transparency in descriptions of and communication regarding AI systems is widely considered desirable. Previous work has suggested that a trade-off exists between greater system transparency and user confusion.
  arXiv  Detail & Related papers  (2021-01-02T11:46:17Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.