VirtualXAI: A User-Centric Framework for Explainability Assessment Leveraging GPT-Generated Personas

• URL: http://arxiv.org/abs/2503.04261v1
• Date: Thu, 06 Mar 2025 09:44:18 GMT
• Title: VirtualXAI: A User-Centric Framework for Explainability Assessment Leveraging GPT-Generated Personas
• Authors: Georgios Makridis, Vasileios Koukos, Georgios Fatouros, Dimosthenis Kyriazis,
• Abstract summary: The demand for eXplainable AI (XAI) has increased to enhance the interpretability, transparency, and trustworthiness of AI models.<n>We propose a framework that integrates quantitative benchmarking with qualitative user assessments through virtual personas.<n>This yields an estimated XAI score and provides tailored recommendations for both the optimal AI model and the XAI method for a given scenario.
• Score: 0.07499722271664146
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In today's data-driven era, computational systems generate vast amounts of data that drive the digital transformation of industries, where Artificial Intelligence (AI) plays a key role. Currently, the demand for eXplainable AI (XAI) has increased to enhance the interpretability, transparency, and trustworthiness of AI models. However, evaluating XAI methods remains challenging: existing evaluation frameworks typically focus on quantitative properties such as fidelity, consistency, and stability without taking into account qualitative characteristics such as satisfaction and interpretability. In addition, practitioners face a lack of guidance in selecting appropriate datasets, AI models, and XAI methods -a major hurdle in human-AI collaboration. To address these gaps, we propose a framework that integrates quantitative benchmarking with qualitative user assessments through virtual personas based on the "Anthology" of backstories of the Large Language Model (LLM). Our framework also incorporates a content-based recommender system that leverages dataset-specific characteristics to match new input data with a repository of benchmarked datasets. This yields an estimated XAI score and provides tailored recommendations for both the optimal AI model and the XAI method for a given scenario.

Related papers

• A Unified Framework for Evaluating the Effectiveness and Enhancing the Transparency of Explainable AI Methods in Real-World Applications [2.0681376988193843]
  "Black box" characteristic of AI models constrains interpretability, transparency, and reliability.<n>This study presents a unified XAI evaluation framework to evaluate correctness, interpretability, robustness, fairness, and completeness of explanations generated by AI models.
  arXiv  Detail & Related papers  (2024-12-05T05:30:10Z)
• AI-Aided Kalman Filters [65.35350122917914]
  The Kalman filter (KF) and its variants are among the most celebrated algorithms in signal processing. Recent developments illustrate the possibility of fusing deep neural networks (DNNs) with classic Kalman-type filtering. This article provides a tutorial-style overview of design approaches for incorporating AI in aiding KF-type algorithms.
  arXiv  Detail & Related papers  (2024-10-16T06:47:53Z)
• XEdgeAI: A Human-centered Industrial Inspection Framework with Data-centric Explainable Edge AI Approach [2.0209172586699173]
  This paper introduces a novel XAI-integrated Visual Quality Inspection framework. Our framework incorporates XAI and the Large Vision Language Model to deliver human-centered interpretability. This approach paves the way for the broader adoption of reliable and interpretable AI tools in critical industrial applications.
  arXiv  Detail & Related papers  (2024-07-16T14:30:24Z)
• Explainable AI for Enhancing Efficiency of DL-based Channel Estimation [1.0136215038345013]
  Support of artificial intelligence based decision-making is a key element in future 6G networks. In such applications, using AI as black-box models is risky and challenging. We propose a novel-based XAI-CHEST framework that is oriented toward channel estimation in wireless communications.
  arXiv  Detail & Related papers  (2024-07-09T16:24:21Z)
• EXACT: Towards a platform for empirically benchmarking Machine Learning model explanation methods [1.6383837447674294]
  This paper brings together various benchmark datasets and novel performance metrics in an initial benchmarking platform. Our datasets incorporate ground truth explanations for class-conditional features. This platform assesses the performance of post-hoc XAI methods in the quality of the explanations they produce.
  arXiv  Detail & Related papers  (2024-05-20T14:16:06Z)
• How much informative is your XAI? A decision-making assessment task to objectively measure the goodness of explanations [53.01494092422942]
  The number and complexity of personalised and user-centred approaches to XAI have rapidly grown in recent years. It emerged that user-centred approaches to XAI positively affect the interaction between users and systems. We propose an assessment task to objectively and quantitatively measure the goodness of XAI systems.
  arXiv  Detail & Related papers  (2023-12-07T15:49:39Z)
• QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
  We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights. We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
  arXiv  Detail & Related papers  (2023-11-06T00:21:44Z)
• Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment [66.91538273487379]
  There is a certain consensus about the need to develop AI applications with a Human-Centric approach. Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes. We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data.
  arXiv  Detail & Related papers  (2023-02-13T16:44:44Z)
• Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI [65.44737844681256]
  A lack of consensus on how to evaluate explainable AI (XAI) hinders the advancement of the field. We argue that one way to close the gap is to develop evaluation methods that account for different user requirements.
  arXiv  Detail & Related papers  (2022-06-22T05:17:33Z)
• Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
  Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations. We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2021-06-14T20:48:48Z)
• A Comparative Approach to Explainable Artificial Intelligence Methods in Application to High-Dimensional Electronic Health Records: Examining the Usability of XAI [0.0]
  XAI aims to produce a demonstrative factor of trust, which for human subjects is achieved through communicative means. The ideology behind trusting a machine to tend towards the livelihood of a human poses an ethical conundrum. XAI methods produce visualization of the feature contribution towards a given models output on both a local and global level.
  arXiv  Detail & Related papers  (2021-03-08T18:15:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.