Using Large Language Models to Compare Explainable Models for Smart Home Human Activity Recognition

• URL: http://arxiv.org/abs/2408.06352v1
• Date: Wed, 24 Jul 2024 12:15:07 GMT
• Title: Using Large Language Models to Compare Explainable Models for Smart Home Human Activity Recognition
• Authors: Michele Fiori, Gabriele Civitarese, Claudio Bettini,
• Abstract summary: This paper proposes an automatic evaluation method using Large Language Models (LLMs) to identify, in a pool of candidates, the best XAI approach for non-expert users. Our preliminary results suggest that LLM evaluation aligns with user surveys.
• Score: 0.3277163122167433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recognizing daily activities with unobtrusive sensors in smart environments enables various healthcare applications. Monitoring how subjects perform activities at home and their changes over time can reveal early symptoms of health issues, such as cognitive decline. Most approaches in this field use deep learning models, which are often seen as black boxes mapping sensor data to activities. However, non-expert users like clinicians need to trust and understand these models' outputs. Thus, eXplainable AI (XAI) methods for Human Activity Recognition have emerged to provide intuitive natural language explanations from these models. Different XAI methods generate different explanations, and their effectiveness is typically evaluated through user surveys, that are often challenging in terms of costs and fairness. This paper proposes an automatic evaluation method using Large Language Models (LLMs) to identify, in a pool of candidates, the best XAI approach for non-expert users. Our preliminary results suggest that LLM evaluation aligns with user surveys.

Related papers

• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning [53.241569810013836]
  We propose a new framework based on large language models (LLMs) and decision Tree reasoning (OCTree) Our key idea is to leverage LLMs' reasoning capabilities to find good feature generation rules without manually specifying the search space. Our empirical results demonstrate that this simple framework consistently enhances the performance of various prediction models.
  arXiv  Detail & Related papers  (2024-06-12T08:31:34Z)
• Modeling User Preferences via Brain-Computer Interfacing [54.3727087164445]
  We use Brain-Computer Interfacing technology to infer users' preferences, their attentional correlates towards visual content, and their associations with affective experience. We link these to relevant applications, such as information retrieval, personalized steering of generative models, and crowdsourcing population estimates of affective experiences.
  arXiv  Detail & Related papers  (2024-05-15T20:41:46Z)
• I-CEE: Tailoring Explanations of Image Classification Models to User Expertise [13.293968260458962]
  We present I-CEE, a framework that provides Image Classification Explanations tailored to User Expertise. I-CEE models the informativeness of the example images to depend on user expertise, resulting in different examples for different users. Experiments with simulated users show that I-CEE improves users' ability to accurately predict the model's decisions.
  arXiv  Detail & Related papers  (2023-12-19T12:26:57Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Explainable Depression Symptom Detection in Social Media [2.677715367737641]
  We propose using transformer-based architectures to detect and explain the appearance of depressive symptom markers in the users' writings. Our natural language explanations enable clinicians to interpret the models' decisions based on validated symptoms.
  arXiv  Detail & Related papers  (2023-10-20T17:05:27Z)
• Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability [70.72426887518517]
  Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
  arXiv  Detail & Related papers  (2023-06-06T14:23:34Z)
• What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps [69.53730499849023]
  We present a user-centered persona concept to evaluate explainable AI (XAI) Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences. Our insights bring an interactive, human-centered XAI closer to practical application.
  arXiv  Detail & Related papers  (2022-10-07T12:51:27Z)
• Human Activity Recognition models using Limited Consumer Device Sensors and Machine Learning [0.0]
  Human activity recognition has grown in popularity with its increase of applications within daily lifestyles and medical environments. This paper presents the findings of different models that are limited to train using sensor data from smartphones and smartwatches. Results show promise for models trained strictly using limited sensor data collected from only smartphones and smartwatches coupled with traditional machine learning concepts and algorithms.
  arXiv  Detail & Related papers  (2022-01-21T06:54:05Z)
• Explainable Activity Recognition for Smart Home Systems [9.909901668370589]
  We build on insights from Explainable Artificial Intelligence (XAI) techniques to develop an explainable activity recognition framework. Our results show that the XAI approach, SHAP, has a 92% success rate in generating sensible explanations. In 83% of sampled scenarios users preferred natural language explanations over a simple activity label.
  arXiv  Detail & Related papers  (2021-05-20T14:35:51Z)
• A Comparative Approach to Explainable Artificial Intelligence Methods in Application to High-Dimensional Electronic Health Records: Examining the Usability of XAI [0.0]
  XAI aims to produce a demonstrative factor of trust, which for human subjects is achieved through communicative means. The ideology behind trusting a machine to tend towards the livelihood of a human poses an ethical conundrum. XAI methods produce visualization of the feature contribution towards a given models output on both a local and global level.
  arXiv  Detail & Related papers  (2021-03-08T18:15:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.