A Human-Centric Take on Model Monitoring
- URL: http://arxiv.org/abs/2206.02868v1
- Date: Mon, 6 Jun 2022 19:47:29 GMT
- Title: A Human-Centric Take on Model Monitoring
- Authors: Murtuza N Shergadwala, Himabindu Lakkaraju, Krishnaram Kenthapadi
- Abstract summary: We identify various human-centric challenges and requirements for model monitoring in real-world applications.
We find the need and the challenge for the model monitoring systems to clarify the impact of the monitoring observations on outcomes.
- Score: 23.006624840045554
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Predictive models are increasingly used to make various consequential
decisions in high-stakes domains such as healthcare, finance, and policy. It
becomes critical to ensure that these models make accurate predictions, are
robust to shifts in the data, do not rely on spurious features, and do not
unduly discriminate against minority groups. To this end, several approaches
spanning various areas such as explainability, fairness, and robustness have
been proposed in recent literature. Such approaches need to be human-centered
as they cater to the understanding of the models to their users. However, there
is a research gap in understanding the human-centric needs and challenges of
monitoring machine learning (ML) models once they are deployed. To fill this
gap, we conducted an interview study with 13 practitioners who have experience
at the intersection of deploying ML models and engaging with customers spanning
domains such as financial services, healthcare, hiring, online retail,
computational advertising, and conversational assistants. We identified various
human-centric challenges and requirements for model monitoring in real-world
applications. Specifically, we found the need and the challenge for the model
monitoring systems to clarify the impact of the monitoring observations on
outcomes. Further, such insights must be actionable, robust, customizable for
domain-specific use cases, and cognitively considerate to avoid information
overload.
Related papers
- Unsupervised Model Diagnosis [49.36194740479798]
This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance.
Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
arXiv Detail & Related papers (2024-10-08T17:59:03Z) - A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Series [17.08674819906415]
We introduce HILAD, a novel framework designed to foster a dynamic and bidirectional collaboration between humans and AI.
Through our visual interface, HILAD empowers domain experts to detect, interpret, and correct unexpected model behaviors at scale.
arXiv Detail & Related papers (2024-05-06T07:44:07Z) - Survey of Social Bias in Vision-Language Models [65.44579542312489]
Survey aims to provide researchers with a high-level insight into the similarities and differences of social bias studies in pre-trained models across NLP, CV, and VL.
The findings and recommendations presented here can benefit the ML community, fostering the development of fairer and non-biased AI models.
arXiv Detail & Related papers (2023-09-24T15:34:56Z) - Safe AI for health and beyond -- Monitoring to transform a health
service [51.8524501805308]
We will assess the infrastructure required to monitor the outputs of a machine learning algorithm.
We will present two scenarios with examples of monitoring and updates of models.
arXiv Detail & Related papers (2023-03-02T17:27:45Z) - A Comprehensive Review of Trends, Applications and Challenges In
Out-of-Distribution Detection [0.76146285961466]
Field of study has emerged, focusing on detecting out-of-distribution data subsets and enabling a more comprehensive generalization.
As many deep learning based models have achieved near-perfect results on benchmark datasets, the need to evaluate these models' reliability and trustworthiness is felt more strongly than ever.
This paper presents a survey that, in addition to reviewing more than 70 papers in this field, presents challenges and directions for future works and offers a unifying look into various types of data shifts and solutions for better generalization.
arXiv Detail & Related papers (2022-09-26T18:13:14Z) - Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards
Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
Key challenge for robotic systems is to figure out the behavior of another agent.
Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally.
We propose equipping robots with the necessary tools to conduct observational studies on people.
arXiv Detail & Related papers (2022-01-27T22:15:56Z) - Individual Explanations in Machine Learning Models: A Survey for
Practitioners [69.02688684221265]
The use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise.
Many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways.
Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models.
arXiv Detail & Related papers (2021-04-09T01:46:34Z) - Combining Graph Neural Networks and Spatio-temporal Disease Models to
Predict COVID-19 Cases in Germany [0.0]
Several experts have called for the necessity to account for human mobility to explain the spread of COVID-19.
Most statistical or epidemiological models cannot directly incorporate unstructured data sources, including data that may encode human mobility.
We propose a trade-off between both research directions and present a novel learning approach that combines the advantages of statistical regression and machine learning models.
arXiv Detail & Related papers (2021-01-03T16:39:00Z) - Plausible Counterfactuals: Auditing Deep Learning Classifiers with
Realistic Adversarial Examples [84.8370546614042]
Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data.
Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model.
Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
arXiv Detail & Related papers (2020-03-25T11:08:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.