Evaluating Bayesian Model Visualisations
- URL: http://arxiv.org/abs/2201.03604v1
- Date: Mon, 10 Jan 2022 19:15:39 GMT
- Title: Evaluating Bayesian Model Visualisations
- Authors: Sebastian Stein (1), John H. Williamson (1) ((1) School of Computing
Science, University of Glasgow, Scotland, United Kingdom)
- Abstract summary: Probabilistic models inform an increasingly broad range of business and policy decisions ultimately made by people.
Recent algorithmic, computational, and software framework development progress facilitate the proliferation of Bayesian probabilistic models.
While they can empower decision makers to explore complex queries and to perform what-if-style conditioning in theory, suitable visualisations and interactive tools are needed to maximise users' comprehension and rational decision making under uncertainty.
- Score: 0.39845810840390733
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Probabilistic models inform an increasingly broad range of business and
policy decisions ultimately made by people. Recent algorithmic, computational,
and software framework development progress facilitate the proliferation of
Bayesian probabilistic models, which characterise unobserved parameters by
their joint distribution instead of point estimates. While they can empower
decision makers to explore complex queries and to perform what-if-style
conditioning in theory, suitable visualisations and interactive tools are
needed to maximise users' comprehension and rational decision making under
uncertainty. In this paper, propose a protocol for quantitative evaluation of
Bayesian model visualisations and introduce a software framework implementing
this protocol to support standardisation in evaluation practice and facilitate
reproducibility. We illustrate the evaluation and analysis workflow on a user
study that explores whether making Boxplots and Hypothetical Outcome Plots
interactive can increase comprehension or rationality and conclude with design
guidelines for researchers looking to conduct similar studies in the future.
Related papers
- A Probabilistic Perspective on Unlearning and Alignment for Large Language Models [48.96686419141881]
We introduce the first formal probabilistic evaluation framework in Large Language Models (LLMs)
We derive novel metrics with high-probability guarantees concerning the output distribution of a model.
Our metrics are application-independent and allow practitioners to make more reliable estimates about model capabilities before deployment.
arXiv Detail & Related papers (2024-10-04T15:44:23Z) - Data Analysis in the Era of Generative AI [56.44807642944589]
This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges.
We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow.
We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
arXiv Detail & Related papers (2024-09-27T06:31:03Z) - Communicating Uncertainty in Machine Learning Explanations: A
Visualization Analytics Approach for Predictive Process Monitoring [0.0]
This study explores how model uncertainty can be effectively communicated in global and local post-hoc explanation approaches.
By combining these two research directions, decision-makers can not only justify the plausibility of explanation-driven actionable insights but also validate their reliability.
arXiv Detail & Related papers (2023-04-12T09:44:32Z) - A review of predictive uncertainty estimation with machine learning [0.0]
We review the topic of predictive uncertainty estimation with machine learning algorithms.
We discuss the related metrics (consistent scoring functions and proper scoring rules) for assessing probabilistic predictions.
The review expedites our understanding on how to develop new algorithms tailored to users' needs.
arXiv Detail & Related papers (2022-09-17T10:36:30Z) - Bayesian Graph Contrastive Learning [55.36652660268726]
We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders.
Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector.
We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
arXiv Detail & Related papers (2021-12-15T01:45:32Z) - Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
We propose a deep co-attention network for multi-view subspace learning.
It aims to extract both the common information and the complementary information in an adversarial setting.
In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
arXiv Detail & Related papers (2021-02-15T18:46:44Z) - (Un)fairness in Post-operative Complication Prediction Models [20.16366948502659]
We consider a real-life example of risk estimation before surgery and investigate the potential for bias or unfairness of a variety of algorithms.
Our approach creates transparent documentation of potential bias so that the users can apply the model carefully.
arXiv Detail & Related papers (2020-11-03T22:11:19Z) - Forethought and Hindsight in Credit Assignment [62.05690959741223]
We work to understand the gains and peculiarities of planning employed as forethought via forward models or as hindsight operating with backward models.
We investigate the best use of models in planning, primarily focusing on the selection of states in which predictions should be (re)-evaluated.
arXiv Detail & Related papers (2020-10-26T16:00:47Z) - Plausible Counterfactuals: Auditing Deep Learning Classifiers with
Realistic Adversarial Examples [84.8370546614042]
Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data.
Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model.
Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
arXiv Detail & Related papers (2020-03-25T11:08:56Z) - Asking the Right Questions: Learning Interpretable Action Models Through
Query Answering [33.08099403894141]
This paper develops a new approach for estimating an interpretable, relational model of a black-box autonomous agent that can plan and act.
Our main contributions are a new paradigm for estimating such models using a minimal query interface with the agent, and a hierarchical querying algorithm that generates an interrogation policy for estimating the agent's internal model.
arXiv Detail & Related papers (2019-12-29T09:05:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.