How much reliable is ChatGPT's prediction on Information Extraction under Input Perturbations?

• URL: http://arxiv.org/abs/2404.05088v1
• Date: Sun, 7 Apr 2024 22:06:19 GMT
• Title: How much reliable is ChatGPT's prediction on Information Extraction under Input Perturbations?
• Authors: Ishani Mondal, Abhilasha Sancheti,
• Abstract summary: We assess the robustness of ChatGPT under input perturbations for one of the most fundamental tasks of Information Extraction (IE) We perform a systematic analysis of ChatGPT's robustness on two NER datasets using both automatic and human evaluation. We find that 1) ChatGPT is more brittle on Drug or Disease replacements (rare entities) compared to the perturbations on widely known Person or Location entities.
• Score: 14.815409733416358
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we assess the robustness (reliability) of ChatGPT under input perturbations for one of the most fundamental tasks of Information Extraction (IE) i.e. Named Entity Recognition (NER). Despite the hype, the majority of the researchers have vouched for its language understanding and generation capabilities; a little attention has been paid to understand its robustness: How the input-perturbations affect 1) the predictions, 2) the confidence of predictions and 3) the quality of rationale behind its prediction. We perform a systematic analysis of ChatGPT's robustness (under both zero-shot and few-shot setup) on two NER datasets using both automatic and human evaluation. Based on automatic evaluation metrics, we find that 1) ChatGPT is more brittle on Drug or Disease replacements (rare entities) compared to the perturbations on widely known Person or Location entities, 2) the quality of explanations for the same entity considerably differ under different types of "Entity-Specific" and "Context-Specific" perturbations and the quality can be significantly improved using in-context learning, and 3) it is overconfident for majority of the incorrect predictions, and hence it could lead to misguidance of the end-users.

Related papers

• Automated Trustworthiness Oracle Generation for Machine Learning Text Classifiers [9.349442247982557]
  Traditional metrics, such as model confidence and accuracy, are insufficient to build human trust in machine learning models. We propose TOKI, which automatically checks whether the prediction-contributing words are related to the predicted class. We also introduce a novel adversarial attack method targeting trustworthiness issues identified by TOKI.
  arXiv  Detail & Related papers  (2024-10-30T03:26:37Z)
• An Evaluation of Explanation Methods for Black-Box Detectors of Machine-Generated Text [2.1439084103679273]
  This study conducts the first systematic evaluation of explanation quality for detectors of machine-generated text. We use a dataset of ChatGPT-generated and human-written documents, and pair predictions of three existing language-model-based detectors with the corresponding explanations. We find that SHAP performs best in terms of faithfulness, stability, and in helping users to predict the detector's behavior.
  arXiv  Detail & Related papers  (2024-08-26T13:14:26Z)
• DEMASQ: Unmasking the ChatGPT Wordsmith [63.8746084667206]
  We propose an effective ChatGPT detector named DEMASQ, which accurately identifies ChatGPT-generated content. Our method addresses two critical factors: (i) the distinct biases in text composition observed in human- and machine-generated content and (ii) the alterations made by humans to evade previous detection methods.
  arXiv  Detail & Related papers  (2023-11-08T21:13:05Z)
• Consistency Analysis of ChatGPT [65.268245109828]
  This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions.
  arXiv  Detail & Related papers  (2023-03-11T01:19:01Z)
• Is ChatGPT a Good NLG Evaluator? A Preliminary Study [121.77986688862302]
  We provide a preliminary meta-evaluation on ChatGPT to show its reliability as an NLG metric. Experimental results show that compared with previous automatic metrics, ChatGPT achieves state-of-the-art or competitive correlation with human judgments. We hope our preliminary study could prompt the emergence of a general-purposed reliable NLG metric.
  arXiv  Detail & Related papers  (2023-03-07T16:57:20Z)
• On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective [67.98821225810204]
  We evaluate the robustness of ChatGPT from the adversarial and out-of-distribution perspective. Results show consistent advantages on most adversarial and OOD classification and translation tasks. ChatGPT shows astounding performance in understanding dialogue-related texts.
  arXiv  Detail & Related papers  (2023-02-22T11:01:20Z)
• The Unreliability of Explanations in Few-Shot In-Context Learning [50.77996380021221]
  We focus on two NLP tasks that involve reasoning over text, namely question answering and natural language inference. We show that explanations judged as good by humans--those that are logically consistent with the input--usually indicate more accurate predictions. We present a framework for calibrating model predictions based on the reliability of the explanations.
  arXiv  Detail & Related papers  (2022-05-06T17:57:58Z)
• Learning to Predict Trustworthiness with Steep Slope Loss [69.40817968905495]
  We study the problem of predicting trustworthiness on real-world large-scale datasets. We observe that the trustworthiness predictors trained with prior-art loss functions are prone to view both correct predictions and incorrect predictions to be trustworthy. We propose a novel steep slope loss to separate the features w.r.t. correct predictions from the ones w.r.t. incorrect predictions by two slide-like curves that oppose each other.
  arXiv  Detail & Related papers  (2021-09-30T19:19:09Z)
• Tribrid: Stance Classification with Neural Inconsistency Detection [9.150728831518459]
  We study the problem of performing automatic stance classification on social media with neural architectures such as BERT. We present a new neural architecture where the input also includes automatically generated negated perspectives over a given claim. The model is jointly learned to make simultaneously multiple predictions, which can be used either to improve the classification of the original perspective or to filter out doubtful predictions.
  arXiv  Detail & Related papers  (2021-09-14T08:13:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.