Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AI

• URL: http://arxiv.org/abs/2409.16978v1
• Date: Wed, 25 Sep 2024 14:40:26 GMT
• Title: Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AI
• Authors: Elisa Nguyen, Johannes Bertram, Evgenii Kortukov, Jean Y. Song, Seong Joon Oh,
• Abstract summary: XAI aims to make AI understandable and useful to humans, but it has been criticised for relying too much on formalism and solutionism. We show how the XAI research community should adopt a top-down, user-focused perspective to ensure user relevance.
• Score: 17.453208581487495
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Explainable AI (XAI) aims to make AI understandable and useful to humans, it has been criticised for relying too much on formalism and solutionism, focusing more on mathematical soundness than user needs. We propose an alternative to this bottom-up approach inspired by design thinking: the XAI research community should adopt a top-down, user-focused perspective to ensure user relevance. We illustrate this with a relatively young subfield of XAI, Training Data Attribution (TDA). With the surge in TDA research and growing competition, the field risks repeating the same patterns of solutionism. We conducted a needfinding study with a diverse group of AI practitioners to identify potential user needs related to TDA. Through interviews (N=10) and a systematic survey (N=31), we uncovered new TDA tasks that are currently largely overlooked. We invite the TDA and XAI communities to consider these novel tasks and improve the user relevance of their research outcomes.

Related papers

• Social Scientists on the Role of AI in Research [2.2665233748698355]
  We present a community-centric study drawing on 284 survey responses and 15 semi-structured interviews with social scientists.<n>We find that the use of AI in research settings has increased significantly among social scientists in step with the widespread popularity of generative AI (genAI)<n>Ethical concerns, particularly around automation bias, deskilling, research misconduct, complex interpretability, and representational harm, are raised in relation to genAI.
  arXiv  Detail & Related papers  (2025-06-12T19:55:36Z)
• Training Data Attribution (TDA): Examining Its Adoption & Use Cases [5.256285764938807]
  This report investigates Training Data Attribution (TDA) and its potential importance to and tractability for reducing extreme risks from AI. We discuss the plausibility and amount of effort it would take to bring existing TDA research efforts from their current state, to an efficient and accurate tool for TDA inference. We list and discuss a series of policies and systems that may be enabled by TDA.
  arXiv  Detail & Related papers  (2025-01-22T05:03:51Z)
• The Ethics of Advanced AI Assistants [53.89899371095332]
  This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user. We consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants.
  arXiv  Detail & Related papers  (2024-04-24T23:18:46Z)
• How much informative is your XAI? A decision-making assessment task to objectively measure the goodness of explanations [53.01494092422942]
  The number and complexity of personalised and user-centred approaches to XAI have rapidly grown in recent years. It emerged that user-centred approaches to XAI positively affect the interaction between users and systems. We propose an assessment task to objectively and quantitatively measure the goodness of XAI systems.
  arXiv  Detail & Related papers  (2023-12-07T15:49:39Z)
• Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge [22.21959942886099]
  We introduce a process-oriented notion of appropriate reliance called critical use that centers the human's ability to situate AI predictions against knowledge that is uniquely available to them but unavailable to the AI model. We conduct a randomized online experiment in a complex social decision-making setting: child maltreatment screening. We find that, by providing participants with accelerated, low-stakes opportunities to practice AI-assisted decision-making, novices came to exhibit patterns of disagreement with AI that resemble those of experienced workers.
  arXiv  Detail & Related papers  (2023-08-30T01:54:31Z)
• Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment [66.91538273487379]
  There is a certain consensus about the need to develop AI applications with a Human-Centric approach. Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes. We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data.
  arXiv  Detail & Related papers  (2023-02-13T16:44:44Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations [18.971689499890363]
  We identify and analyze 97core papers with human-based XAI evaluations over the past five years. Our research shows that XAI is spreading more rapidly in certain application domains, such as recommender systems. We propose practical guidelines on designing and conducting user studies for XAI researchers and practitioners.
  arXiv  Detail & Related papers  (2022-10-20T20:53:00Z)
• "Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction [22.00514030715286]
  We conducted a study of a real-world AI application via interviews with 20 end-users of Merlin, a bird-identification app. We found that people express a need for practically useful information that can improve their collaboration with the AI system. We also assessed end-users' perceptions of existing XAI approaches, finding that they prefer part-based explanations.
  arXiv  Detail & Related papers  (2022-10-02T20:17:11Z)
• Transcending XAI Algorithm Boundaries through End-User-Inspired Design [27.864338632191608]
  Lacking explainability-focused functional support for end users may hinder the safe and responsible use of AI in high-stakes domains. Our work shows that grounding the technical problem in end users' use of XAI can inspire new research questions. Such end-user-inspired research questions have the potential to promote social good by democratizing AI and ensuring the responsible use of AI in critical domains.
  arXiv  Detail & Related papers  (2022-08-18T09:44:51Z)
• Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI [65.44737844681256]
  A lack of consensus on how to evaluate explainable AI (XAI) hinders the advancement of the field. We argue that one way to close the gap is to develop evaluation methods that account for different user requirements.
  arXiv  Detail & Related papers  (2022-06-22T05:17:33Z)
• Human-Centered Explainable AI (XAI): From Algorithms to User Experiences [29.10123472973571]
  explainable AI (XAI) has produced a vast collection of algorithms in recent years. The field is starting to embrace inter-disciplinary perspectives and human-centered approaches.
  arXiv  Detail & Related papers  (2021-10-20T21:33:46Z)
• A User-Centred Framework for Explainable Artificial Intelligence in Human-Robot Interaction [70.11080854486953]
  We propose a user-centred framework for XAI that focuses on its social-interactive aspect. The framework aims to provide a structure for interactive XAI solutions thought for non-expert users.
  arXiv  Detail & Related papers  (2021-09-27T09:56:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.