Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena

• URL: http://arxiv.org/abs/2206.05487v3
• Date: Mon, 15 Jul 2024 16:36:50 GMT
• Title: Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena
• Authors: Timo Freiesleben, Gunnar König, Christoph Molnar, Alvaro Tejero-Cantero,
• Abstract summary: Interpretable machine learning offers a solution by analyzing models holistically to derive interpretations. Current IML research is focused on auditing ML models rather than leveraging them for scientific inference. We present a framework for designing IML methods-termed 'property descriptors' that illuminate not just the model, but also the phenomenon it represents.
• Score: 4.312340306206884
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To learn about real world phenomena, scientists have traditionally used models with clearly interpretable elements. However, modern machine learning (ML) models, while powerful predictors, lack this direct elementwise interpretability (e.g. neural network weights). Interpretable machine learning (IML) offers a solution by analyzing models holistically to derive interpretations. Yet, current IML research is focused on auditing ML models rather than leveraging them for scientific inference. Our work bridges this gap, presenting a framework for designing IML methods-termed 'property descriptors' -- that illuminate not just the model, but also the phenomenon it represents. We demonstrate that property descriptors, grounded in statistical learning theory, can effectively reveal relevant properties of the joint probability distribution of the observational data. We identify existing IML methods suited for scientific inference and provide a guide for developing new descriptors with quantified epistemic uncertainty. Our framework empowers scientists to harness ML models for inference, and provides directions for future IML research to support scientific understanding.

Related papers

• Unveiling LLMs: The Evolution of Latent Representations in a Temporal Knowledge Graph [15.129079475322637]
  Large Language Models (LLMs) demonstrate an impressive capacity to recall a vast range of common factual knowledge information. We propose an end-to-end framework that jointly decodes the factual knowledge embedded in the latent space of LLMs. We showcase our framework with local and global interpretability analyses using two claim verification datasets.
  arXiv  Detail & Related papers  (2024-04-04T17:45:59Z)
• Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension [63.330262740414646]
  We study how to characterize and predict the truthfulness of texts generated from large language models (LLMs) We suggest investigating internal activations and quantifying LLM's truthfulness using the local intrinsic dimension (LID) of model activations.
  arXiv  Detail & Related papers  (2024-02-28T04:56:21Z)
• Large Language Model-Based Interpretable Machine Learning Control in Building Energy Systems [3.580636644178055]
  This paper investigates and explores Interpretable Machine Learning (IML), a branch of Machine Learning (ML) that enhances transparency and understanding of models and their inferences. We develop an innovative framework that combines the principles of Shapley values and the in-context learning feature of Large Language Models (LLMs) The paper presents a case study to demonstrate the feasibility of the developed IML framework for model predictive control-based precooling under demand response events in a virtual testbed.
  arXiv  Detail & Related papers  (2024-02-14T21:19:33Z)
• A Mass-Conserving-Perceptron for Machine Learning-Based Modeling of Geoscientific Systems [1.1510009152620668]
  We propose a physically-interpretable Mass Conserving Perceptron (MCP) as a way to bridge the gap between PC-based and ML-based modeling approaches. The MCP exploits the inherent isomorphism between the directed graph structures underlying both PC models and GRNNs to explicitly represent the mass-conserving nature of physical processes.
  arXiv  Detail & Related papers  (2023-10-12T18:09:33Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• Evaluating and Explaining Large Language Models for Code Using Syntactic Structures [74.93762031957883]
  This paper introduces ASTxplainer, an explainability method specific to Large Language Models for code. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes. We perform an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects.
  arXiv  Detail & Related papers  (2023-08-07T18:50:57Z)
• GAM(e) changer or not? An evaluation of interpretable machine learning models based on additive model constraints [5.783415024516947]
  This paper investigates a series of intrinsically interpretable machine learning models. We evaluate the prediction qualities of five GAMs as compared to six traditional ML models.
  arXiv  Detail & Related papers  (2022-04-19T20:37:31Z)
• Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement [75.00655434905417]
  Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex machine learning (ML) models. This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models. We show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning.
  arXiv  Detail & Related papers  (2022-03-15T15:44:28Z)
• Hessian-based toolbox for reliable and interpretable machine learning in physics [58.720142291102135]
  We present a toolbox for interpretability and reliability, extrapolation of the model architecture. It provides a notion of the influence of the input data on the prediction at a given test point, an estimation of the uncertainty of the model predictions, and an agnostic score for the model predictions. Our work opens the road to the systematic use of interpretability and reliability methods in ML applied to physics and, more generally, science.
  arXiv  Detail & Related papers  (2021-08-04T16:32:59Z)
• Understanding Interpretability by generalized distillation in Supervised Classification [3.5473853445215897]
  Recent interpretation strategies focus on human understanding of the underlying decision mechanisms of the complex Machine Learning models. We propose an interpretation-by-distillation formulation that is defined relative to other ML models. We evaluate our proposed framework on the MNIST, Fashion-MNIST and Stanford40 datasets.
  arXiv  Detail & Related papers  (2020-12-05T17:42:50Z)
• Towards Interpretable Deep Learning Models for Knowledge Tracing [62.75876617721375]
  We propose to adopt the post-hoc method to tackle the interpretability issue for deep learning based knowledge tracing (DLKT) models. Specifically, we focus on applying the layer-wise relevance propagation (LRP) method to interpret RNN-based DLKT model. Experiment results show the feasibility using the LRP method for interpreting the DLKT model's predictions.
  arXiv  Detail & Related papers  (2020-05-13T04:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.