TRIED: Truly Innovative and Effective AI Detection Benchmark, developed by WITNESS

• URL: http://arxiv.org/abs/2504.21489v2
• Date: Thu, 01 May 2025 13:38:27 GMT
• Title: TRIED: Truly Innovative and Effective AI Detection Benchmark, developed by WITNESS
• Authors: Shirin Anlen, Zuzanna Wojciak,
• Abstract summary: WITNESS introduces the Truly Innovative and Effective AI Detection (TRIED) Benchmark.<n>The report outlines how detection tools must evolve to become truly innovative and relevant.<n>It offers practical guidance for developers, policy actors, and standards bodies to design accountable, transparent, and user-centered detection solutions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The proliferation of generative AI and deceptive synthetic media threatens the global information ecosystem, especially across the Global Majority. This report from WITNESS highlights the limitations of current AI detection tools, which often underperform in real-world scenarios due to challenges related to explainability, fairness, accessibility, and contextual relevance. In response, WITNESS introduces the Truly Innovative and Effective AI Detection (TRIED) Benchmark, a new framework for evaluating detection tools based on their real-world impact and capacity for innovation. Drawing on frontline experiences, deceptive AI cases, and global consultations, the report outlines how detection tools must evolve to become truly innovative and relevant by meeting diverse linguistic, cultural, and technological contexts. It offers practical guidance for developers, policy actors, and standards bodies to design accountable, transparent, and user-centered detection solutions, and incorporate sociotechnical considerations into future AI standards, procedures and evaluation frameworks. By adopting the TRIED Benchmark, stakeholders can drive innovation, safeguard public trust, strengthen AI literacy, and contribute to a more resilient global information credibility.

Related papers

• Information Retrieval in the Age of Generative AI: The RGB Model [77.96475639967431]
  This paper presents a novel quantitative approach to shed light on the complex information dynamics arising from the growing use of generative AI tools. We propose a model to characterize the generation, indexing, and dissemination of information in response to new topics. Our findings suggest that the rapid pace of generative AI adoption, combined with increasing user reliance, can outpace human verification, escalating the risk of inaccurate information proliferation.
  arXiv  Detail & Related papers  (2025-04-29T10:21:40Z)
• AIJIM: A Scalable Model for Real-Time AI in Environmental Journalism [0.0]
  AIJIM is a framework for integrating real-time AI into environmental journalism.<n>It was validated in a 2024 pilot on the island of Mallorca.<n>It achieved 85.4% detection accuracy and 89.7% agreement with expert annotations.
  arXiv  Detail & Related papers  (2025-03-19T19:00:24Z)
• Toward Agentic AI: Generative Information Retrieval Inspired Intelligent Communications and Networking [87.82985288731489]
  Agentic AI has emerged as a key paradigm for intelligent communications and networking.<n>This article emphasizes the role of knowledge acquisition, processing, and retrieval in agentic AI for telecom systems.
  arXiv  Detail & Related papers  (2025-02-24T06:02:25Z)
• On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspective [333.9220561243189]
  Generative Foundation Models (GenFMs) have emerged as transformative tools.<n>Their widespread adoption raises critical concerns regarding trustworthiness across dimensions.<n>This paper presents a comprehensive framework to address these challenges through three key contributions.
  arXiv  Detail & Related papers  (2025-02-20T06:20:36Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey [92.36487127683053]
  Retrieval-Augmented Generation (RAG) is an advanced technique designed to address the challenges of Artificial Intelligence-Generated Content (AIGC) RAG provides reliable and up-to-date external knowledge, reduces hallucinations, and ensures relevant context across a wide range of tasks. Despite RAG's success and potential, recent studies have shown that the RAG paradigm also introduces new risks, including privacy concerns, adversarial attacks, and accountability issues.
  arXiv  Detail & Related papers  (2025-02-08T06:50:47Z)
• Bridging the Communication Gap: Evaluating AI Labeling Practices for Trustworthy AI Development [41.64451715899638]
  High-level AI labels, inspired by frameworks like EU energy labels, have been proposed to make the properties of AI models more transparent. This study evaluates AI labeling through qualitative interviews along four key research questions.
  arXiv  Detail & Related papers  (2025-01-21T06:00:14Z)
• A Unified Framework for Evaluating the Effectiveness and Enhancing the Transparency of Explainable AI Methods in Real-World Applications [2.0681376988193843]
  "Black box" characteristic of AI models constrains interpretability, transparency, and reliability.<n>This study presents a unified XAI evaluation framework to evaluate correctness, interpretability, robustness, fairness, and completeness of explanations generated by AI models.
  arXiv  Detail & Related papers  (2024-12-05T05:30:10Z)
• Responsible AI in Open Ecosystems: Reconciling Innovation with Risk Assessment and Disclosure [4.578401882034969]
  We focus on how model performance evaluation may inform or inhibit probing of model limitations, biases, and other risks. Our findings can inform AI providers and legal scholars in designing interventions and policies that preserve open-source innovation while incentivizing ethical uptake.
  arXiv  Detail & Related papers  (2024-09-27T19:09:40Z)
• Deepfakes, Misinformation, and Disinformation in the Era of Frontier AI, Generative AI, and Large AI Models [7.835719708227145]
  Deepfakes and the spread of m/disinformation have emerged as formidable threats to the integrity of information ecosystems worldwide. We highlight the mechanisms through which generative AI based on large models (LM-based GenAI) craft seemingly convincing yet fabricated contents. We introduce an integrated framework that combines advanced detection algorithms, cross-platform collaboration, and policy-driven initiatives.
  arXiv  Detail & Related papers  (2023-11-29T06:47:58Z)
• An interdisciplinary conceptual study of Artificial Intelligence (AI) for helping benefit-risk assessment practices: Towards a comprehensive qualification matrix of AI programs and devices (pre-print 2020) [55.41644538483948]
  This paper proposes a comprehensive analysis of existing concepts coming from different disciplines tackling the notion of intelligence. The aim is to identify shared notions or discrepancies to consider for qualifying AI systems.
  arXiv  Detail & Related papers  (2021-05-07T12:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.