Can ChatGPT Make Explanatory Inferences? Benchmarks for Abductive Reasoning

• URL: http://arxiv.org/abs/2404.18982v1
• Date: Mon, 29 Apr 2024 15:19:05 GMT
• Title: Can ChatGPT Make Explanatory Inferences? Benchmarks for Abductive Reasoning
• Authors: Paul Thagard,
• Abstract summary: This paper proposes a set of benchmarks for assessing the ability of AI programs to perform explanatory inference. Tests on the benchmarks reveal that ChatGPT performs creative and evaluative inferences in many domains. Claims that ChatGPT and similar models are incapable of explanation, understanding, causal reasoning, meaning, and creativity are rebutted.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Explanatory inference is the creation and evaluation of hypotheses that provide explanations, and is sometimes known as abduction or abductive inference. Generative AI is a new set of artificial intelligence models based on novel algorithms for generating text, images, and sounds. This paper proposes a set of benchmarks for assessing the ability of AI programs to perform explanatory inference, and uses them to determine the extent to which ChatGPT, a leading generative AI model, is capable of making explanatory inferences. Tests on the benchmarks reveal that ChatGPT performs creative and evaluative inferences in many domains, although it is limited to verbal and visual modalities. Claims that ChatGPT and similar models are incapable of explanation, understanding, causal reasoning, meaning, and creativity are rebutted.

Related papers

• Validity Arguments For Constructed Response Scoring Using Generative Artificial Intelligence Applications [0.0]
  generative AI is particularly appealing because it reduces the effort required for handcrafting features in traditional AI scoring. We compare the validity evidence needed in scoring systems using human ratings, feature-based natural language processing AI scoring engines, and generative AI.
  arXiv  Detail & Related papers  (2025-01-04T16:59:29Z)
• SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation [19.833055725825883]
  We introduce Spectrogram Perturbation for Explainable Speech-to-text Generation (SPES) SPES provides explanations for each predicted token based on both the input spectrogram and the previously generated tokens. Extensive evaluation on speech recognition and translation demonstrates that SPES generates explanations that are faithful and plausible to humans.
  arXiv  Detail & Related papers  (2024-11-03T23:02:30Z)
• Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Text [4.902089836908786]
  WhosAI is a triplet-network contrastive learning framework designed to predict whether a given input text has been generated by humans or AI. We show that our proposed framework achieves outstanding results in both the Turing Test and Authorship tasks.
  arXiv  Detail & Related papers  (2024-07-12T15:44:56Z)
• Explaining Text Similarity in Transformer Models [52.571158418102584]
  Recent advances in explainable AI have made it possible to mitigate limitations by leveraging improved explanations for Transformers. We use BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, to investigate which feature interactions drive similarity in NLP models. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.
  arXiv  Detail & Related papers  (2024-05-10T17:11:31Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Explaining Hate Speech Classification with Model Agnostic Methods [0.9990687944474738]
  The research goal of this paper is to bridge the gap between hate speech prediction and the explanations generated by the system to support its decision. This has been achieved by first predicting the classification of a text and then providing a posthoc, model agnostic and surrogate interpretability approach.
  arXiv  Detail & Related papers  (2023-05-30T19:52:56Z)
• Generative AI: Implications and Applications for Education [0.0]
  The launch of ChatGPT in November 2022 precipitated a panic among some educators while prompting qualified enthusiasm from others. Under the umbrella term Generative AI, ChatGPT is an example of a range of technologies for the delivery of computer-generated text, image, and other digitized media.
  arXiv  Detail & Related papers  (2023-05-12T16:52:38Z)
• Visual Abductive Reasoning [85.17040703205608]
  Abductive reasoning seeks the likeliest possible explanation for partial observations. We propose a new task and dataset, Visual Abductive Reasoning ( VAR), for examining abductive reasoning ability of machine intelligence in everyday visual situations.
  arXiv  Detail & Related papers  (2022-03-26T10:17:03Z)
• Local Explanation of Dialogue Response Generation [77.68077106724522]
  Local explanation of response generation (LERG) is proposed to gain insights into the reasoning process of a generation model. LERG views the sequence prediction as uncertainty estimation of a human response and then creates explanations by perturbing the input and calculating the certainty change over the human response. Our results show that our method consistently improves other widely used methods on proposed automatic- and human- evaluation metrics for this new task by 4.4-12.8%.
  arXiv  Detail & Related papers  (2021-06-11T17:58:36Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.