TAI Scan Tool: A RAG-Based Tool With Minimalistic Input for Trustworthy AI Self-Assessment

• URL: http://arxiv.org/abs/2507.17514v1
• Date: Wed, 23 Jul 2025 13:51:23 GMT
• Title: TAI Scan Tool: A RAG-Based Tool With Minimalistic Input for Trustworthy AI Self-Assessment
• Authors: Athanasios Davvetas, Xenia Ziouvelou, Ypatia Dami, Alexis Kaponis, Konstantina Giouvanopoulou, Michael Papademas,
• Abstract summary: This paper introduces the TAI Scan Tool, a RAG-based TAI self-assessment tool with minimalistic input.<n>The current version supports the legal TAI assessment, with a particular emphasis on facilitating compliance with the AI Act.<n>The assessment output of the system includes insight regarding the risk-level of the AI system according to the AI Act, while at the same time retrieving relevant articles to aid with compliance and notify on their obligations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces the TAI Scan Tool, a RAG-based TAI self-assessment tool with minimalistic input. The current version of the tool supports the legal TAI assessment, with a particular emphasis on facilitating compliance with the AI Act. It involves a two-step approach with a pre-screening and an assessment phase. The assessment output of the system includes insight regarding the risk-level of the AI system according to the AI Act, while at the same time retrieving relevant articles to aid with compliance and notify on their obligations. Our qualitative evaluation using use-case scenarios yields promising results, correctly predicting risk levels while retrieving relevant articles across three distinct semantic groups. Furthermore, interpretation of results shows that the tool's reasoning relies on comparison with the setting of high-risk systems, a behaviour attributed to their deployment requiring careful consideration, and therefore frequently presented within the AI Act.

Related papers

• Efficacy of AI RAG Tools for Complex Information Extraction and Data Annotation Tasks: A Case Study Using Banks Public Disclosures [0.0]
  We replicate an existing, challenging real-world annotation task with complex multi-part criteria on thousands of pages of public disclosure documents.<n>We find that when extrapolated to the full task, these methods could save up to 268 hours compared to the human-only approach.
  arXiv  Detail & Related papers  (2025-07-28T22:06:11Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Beyond Detection: Designing AI-Resilient Assessments with Automated Feedback Tool to Foster Critical Thinking [0.0]
  This research proposes a proactive, AI-resilient solution based on assessment design rather than detection.<n>It introduces a web-based Python tool that integrates Bloom's taxonomy with advanced natural language processing techniques.<n>It helps educators determine whether a task targets lower-order thinking such as recall and summarization or higher-order skills such as analysis, evaluation, and creation.
  arXiv  Detail & Related papers  (2025-03-30T23:13:00Z)
• General Scales Unlock AI Evaluation with Explanatory and Predictive Power [57.7995945974989]
  benchmarking has guided progress in AI, but it has offered limited explanatory and predictive power for general-purpose AI systems.<n>We introduce general scales for AI evaluation that can explain what common AI benchmarks really measure.<n>Our fully-automated methodology builds on 18 newly-crafted rubrics that place instance demands on general scales that do not saturate.
  arXiv  Detail & Related papers  (2025-03-09T01:13:56Z)
• Measuring AI agent autonomy: Towards a scalable approach with code inspection [8.344207672507334]
  We introduce a code-based assessment of autonomy that eliminates the need to run an AI agent to perform specific tasks.<n>We demonstrate this approach with the AutoGen framework and select applications.
  arXiv  Detail & Related papers  (2025-02-21T04:58:40Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• Interpretable Rule-Based System for Radar-Based Gesture Sensing: Enhancing Transparency and Personalization in AI [2.99664686845172]
  We introduce MIRA, a transparent and interpretable multi-class rule-based algorithm tailored for radar-based gesture detection. We showcase the system's adaptability through personalized rule sets that calibrate to individual user behavior, offering a user-centric AI experience. Our research underscores MIRA's ability to deliver both high interpretability and performance and emphasizes the potential for broader adoption of interpretable AI in safety-critical applications.
  arXiv  Detail & Related papers  (2024-09-30T16:40:27Z)
• Training Compute Thresholds: Features and Functions in AI Regulation [0.7234862895932991]
  Regulators in the US and EU are using thresholds based on training compute to identify GPAI models that may pose risks of large-scale societal harm. We argue that training compute currently is the most suitable metric to identify GPAI models that deserve regulatory oversight and further scrutiny. As GPAI technology and market structures evolve, regulators should update compute thresholds and complement them with other metrics into regulatory review processes.
  arXiv  Detail & Related papers  (2024-05-17T14:10:24Z)
• Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
  The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs) It examines the challenges introduced by AI components and the impact on testing procedures. The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
  arXiv  Detail & Related papers  (2024-02-21T08:29:42Z)
• Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation [160.07938471250048]
  Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics. We develop strong-performing automatic metrics for reference-based summarization evaluation.
  arXiv  Detail & Related papers  (2023-03-07T02:49:50Z)
• Multisource AI Scorecard Table for System Evaluation [3.74397577716445]
  The paper describes a Multisource AI Scorecard Table (MAST) that provides the developer and user of an artificial intelligence (AI)/machine learning (ML) system with a standard checklist. The paper explores how the analytic tradecraft standards outlined in Intelligence Community Directive (ICD) 203 can provide a framework for assessing the performance of an AI system.
  arXiv  Detail & Related papers  (2021-02-08T03:37:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.