Towards Integrating Personal Knowledge into Test-Time Predictions

• URL: http://arxiv.org/abs/2406.08636v1
• Date: Wed, 12 Jun 2024 20:47:17 GMT
• Title: Towards Integrating Personal Knowledge into Test-Time Predictions
• Authors: Isaac Lage, Sonali Parbhoo, Finale Doshi-Velez,
• Abstract summary: Machine learning models can make decisions based on large amounts of data, but they can be missing personal knowledge available to human users about whom predictions are made. In this work, we introduce the problem of human feature integration, which provides a way to incorporate important personal-knowledge from users without domain expertise into ML predictions.
• Score: 23.303750906345844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning (ML) models can make decisions based on large amounts of data, but they can be missing personal knowledge available to human users about whom predictions are made. For example, a model trained to predict psychiatric outcomes may know nothing about a patient's social support system, and social support may look different for different patients. In this work, we introduce the problem of human feature integration, which provides a way to incorporate important personal-knowledge from users without domain expertise into ML predictions. We characterize this problem through illustrative user stories and comparisons to existing approaches; we formally describe this problem in a way that paves the ground for future technical solutions; and we provide a proof-of-concept study of a simple version of a solution to this problem in a semi-realistic setting.

Related papers

• Which Client is Reliable?: A Reliable and Personalized Prompt-based Federated Learning for Medical Image Question Answering [51.26412822853409]
  We present a novel personalized federated learning (pFL) method for medical visual question answering (VQA) models. Our method introduces learnable prompts into a Transformer architecture to efficiently train it on diverse medical datasets without massive computational costs.
  arXiv  Detail & Related papers  (2024-10-23T00:31:17Z)
• Learning Multimodal Cues of Children's Uncertainty [19.349368123567658]
  We present a dataset annotated in collaboration with developmental and cognitive psychologists for the purpose of studying nonverbal cues of uncertainty. We then present an analysis of the data, studying different roles of uncertainty and its relationship with task difficulty and performance. Lastly, we present a multimodal machine learning model that can predict uncertainty given a real-time video clip of a participant.
  arXiv  Detail & Related papers  (2024-10-17T21:46:00Z)
• Unified Uncertainty Estimation for Cognitive Diagnosis Models [70.46998436898205]
  We propose a unified uncertainty estimation approach for a wide range of cognitive diagnosis models. We decompose the uncertainty of diagnostic parameters into data aspect and model aspect. Our method is effective and can provide useful insights into the uncertainty of cognitive diagnosis.
  arXiv  Detail & Related papers  (2024-03-09T13:48:20Z)
• Modeling Uncertainty in Personalized Emotion Prediction with Normalizing Flows [6.32047610997385]
  This work proposes a novel approach to capture the uncertainty of the forecast using conditional Normalizing Flows. We validated our method on three challenging, subjective NLP tasks, including emotion recognition and hate speech. The information brought by the developed methods makes it possible to build hybrid models whose effectiveness surpasses classic solutions.
  arXiv  Detail & Related papers  (2023-12-10T23:21:41Z)
• Human Uncertainty in Concept-Based AI Systems [37.82747673914624]
  We study human uncertainty in the context of concept-based AI systems. We show that training with uncertain concept labels may help mitigate weaknesses in concept-based systems.
  arXiv  Detail & Related papers  (2023-03-22T19:17:57Z)
• What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps [69.53730499849023]
  We present a user-centered persona concept to evaluate explainable AI (XAI) Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences. Our insights bring an interactive, human-centered XAI closer to practical application.
  arXiv  Detail & Related papers  (2022-10-07T12:51:27Z)
• A Human-Centric Take on Model Monitoring [23.006624840045554]
  We identify various human-centric challenges and requirements for model monitoring in real-world applications. We find the need and the challenge for the model monitoring systems to clarify the impact of the monitoring observations on outcomes.
  arXiv  Detail & Related papers  (2022-06-06T19:47:29Z)
• Enhancing Human-Machine Teaming for Medical Prognosis Through Neural Ordinary Differential Equations (NODEs) [0.0]
  A key barrier to the full realization of Machine Learning's potential in medical prognoses is technology acceptance. Recent efforts to produce explainable AI (XAI) have made progress in improving the interpretability of some ML models. We propose a novel ML architecture to enhance human understanding and encourage acceptability.
  arXiv  Detail & Related papers  (2021-02-08T10:52:23Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)
• When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making [68.19284302320146]
  We carry out user studies to assess how people with differing levels of expertise respond to different types of predictive uncertainty. We found that showing posterior predictive distributions led to smaller disagreements with the ML model's predictions. This suggests that posterior predictive distributions can potentially serve as useful decision aids which should be used with caution and take into account the type of distribution and the expertise of the human.
  arXiv  Detail & Related papers  (2020-11-12T02:23:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.