To what extent should we trust AI models when they extrapolate?

• URL: http://arxiv.org/abs/2201.11260v1
• Date: Thu, 27 Jan 2022 01:27:11 GMT
• Title: To what extent should we trust AI models when they extrapolate?
• Authors: Roozbeh Yousefzadeh and Xuenan Cao
• Abstract summary: We show that models extrapolate frequently; the extent of extrapolation varies and can be socially consequential. This paper investigates several social applications of AI, showing how models extrapolate without notice.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many applications affecting human lives rely on models that have come to be known under the umbrella of machine learning and artificial intelligence. These AI models are usually complicated mathematical functions that map from an input space to an output space. Stakeholders are interested to know the rationales behind models' decisions and functional behavior. We study this functional behavior in relation to the data used to create the models. On this topic, scholars have often assumed that models do not extrapolate, i.e., they learn from their training samples and process new input by interpolation. This assumption is questionable: we show that models extrapolate frequently; the extent of extrapolation varies and can be socially consequential. We demonstrate that extrapolation happens for a substantial portion of datasets more than one would consider reasonable. How can we trust models if we do not know whether they are extrapolating? Given a model trained to recommend clinical procedures for patients, can we trust the recommendation when the model considers a patient older or younger than all the samples in the training set? If the training set is mostly Whites, to what extent can we trust its recommendations about Black and Hispanic patients? Which dimension (race, gender, or age) does extrapolation happen? Even if a model is trained on people of all races, it still may extrapolate in significant ways related to race. The leading question is, to what extent can we trust AI models when they process inputs that fall outside their training set? This paper investigates several social applications of AI, showing how models extrapolate without notice. We also look at different sub-spaces of extrapolation for specific individuals subject to AI models and report how these extrapolations can be interpreted, not mathematically, but from a humanistic point of view.

Related papers

• Great Models Think Alike and this Undermines AI Oversight [47.7725284401918]
  We study how model similarity affects both aspects of AI oversight. We propose a probabilistic metric for LM similarity based on overlap in model mistakes. Our work underscores the importance of reporting and correcting for model similarity.
  arXiv  Detail & Related papers  (2025-02-06T18:56:01Z)
• AutoElicit: Using Large Language Models for Expert Prior Elicitation in Predictive Modelling [53.54623137152208]
  We introduce AutoElicit to extract knowledge from large language models and construct priors for predictive models. We show these priors are informative and can be refined using natural language. We find that AutoElicit yields priors that can substantially reduce error over uninformative priors, using fewer labels, and consistently outperform in-context learning.
  arXiv  Detail & Related papers  (2024-11-26T10:13:39Z)
• Position: Stop Making Unscientific AGI Performance Claims [6.343515088115924]
  Developments in the field of Artificial Intelligence (AI) have created a 'perfect storm' for observing'sparks' of Artificial General Intelligence (AGI) We argue and empirically demonstrate that the finding of meaningful patterns in latent spaces of models cannot be seen as evidence in favor of AGI. We conclude that both the methodological setup and common public image of AI are ideal for the misinterpretation that correlations between model representations and some variables of interest are 'caused' by the model's understanding of underlying 'ground truth' relationships.
  arXiv  Detail & Related papers  (2024-02-06T12:42:21Z)
• Learning Defect Prediction from Unrealistic Data [57.53586547895278]
  Pretrained models of code have become popular choices for code understanding and generation tasks. Such models tend to be large and require commensurate volumes of training data. It has become popular to train models with far larger but less realistic datasets, such as functions with artificially injected bugs. Models trained on such data tend to only perform well on similar data, while underperforming on real world programs.
  arXiv  Detail & Related papers  (2023-11-02T01:51:43Z)
• Scaling Laws Do Not Scale [54.72120385955072]
  Recent work has argued that as the size of a dataset increases, the performance of a model trained on that dataset will increase. We argue that this scaling law relationship depends on metrics used to measure performance that may not correspond with how different groups of people perceive the quality of models' output. Different communities may also have values in tension with each other, leading to difficult, potentially irreconcilable choices about metrics used for model evaluations.
  arXiv  Detail & Related papers  (2023-07-05T15:32:21Z)
• On Inductive Biases for Machine Learning in Data Constrained Settings [0.0]
  This thesis explores a different answer to the problem of learning expressive models in data constrained settings. Instead of relying on big datasets to learn neural networks, we will replace some modules by known functions reflecting the structure of the data. Our approach falls under the hood of "inductive biases", which can be defined as hypothesis on the data at hand restricting the space of models to explore.
  arXiv  Detail & Related papers  (2023-02-21T14:22:01Z)
• Constructing Effective Machine Learning Models for the Sciences: A Multidisciplinary Perspective [77.53142165205281]
  We show how flexible non-linear solutions will not always improve upon manually adding transforms and interactions between variables to linear regression models. We discuss how to recognize this before constructing a data-driven model and how such analysis can help us move to intrinsically interpretable regression models.
  arXiv  Detail & Related papers  (2022-11-21T17:48:44Z)
• Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning [92.89846887298852]
  Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data. Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
  arXiv  Detail & Related papers  (2022-10-11T10:20:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.