What has ChatGPT read? The origins of archaeological citations used by a generative artificial intelligence application

• URL: http://arxiv.org/abs/2308.03301v1
• Date: Mon, 7 Aug 2023 05:06:35 GMT
• Title: What has ChatGPT read? The origins of archaeological citations used by a generative artificial intelligence application
• Authors: Dirk HR Spennemann
• Abstract summary: This paper tested what archaeological literature appears to have been included in ChatGPT's training phase. While ChatGPT offered seemingly pertinent references, a large percentage proved to be fictitious. It can be shown that all references provided by ChatGPT that were found to be genuine have also been cited on Wikipedia pages.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The public release of ChatGPT has resulted in considerable publicity and has led to wide-spread discussion of the usefulness and capabilities of generative AI language models. Its ability to extract and summarise data from textual sources and present them as human-like contextual responses makes it an eminently suitable tool to answer questions users might ask. This paper tested what archaeological literature appears to have been included in ChatGPT's training phase. While ChatGPT offered seemingly pertinent references, a large percentage proved to be fictitious. Using cloze analysis to make inferences on the sources 'memorised' by a generative AI model, this paper was unable to prove that ChatGPT had access to the full texts of the genuine references. It can be shown that all references provided by ChatGPT that were found to be genuine have also been cited on Wikipedia pages. This strongly indicates that the source base for at least some of the data is found in those pages. The implications of this in relation to data quality are discussed.

Related papers

• Chatbot-supported Thesis Writing: An Autoethnographic Report [0.0]
  ChatGPT might be applied to formats that require learners to generate text, such as bachelor theses or student research papers. ChatGPT is to be valued as a beneficial tool in thesis writing. However, writing a conclusive thesis still requires the learner's meaningful engagement.
  arXiv  Detail & Related papers  (2023-10-14T09:09:26Z)
• ChatGPT Hallucinates when Attributing Answers [27.63520311803786]
  We investigate how different prompts impact answers and evidence. We find that ChatGPT provides correct or partially correct answers in about half of the cases. But its suggested references only exist 14% of the times.
  arXiv  Detail & Related papers  (2023-09-17T23:49:12Z)
• Do androids dream of fictional references? A bibliographic dialogue with ChatGPT3.5 [0.0]
  This article focuses on references generated by the ChatGPT3.5 tool. We explored six different themes and analyzed a sample of references generated by the model, in French and English.
  arXiv  Detail & Related papers  (2023-09-04T08:11:59Z)
• Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text [48.36706154871577]
  We introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts) It diverges from extant corpora by comprising pairs of human-written and ChatGPT-polished abstracts instead of purely ChatGPT-generated texts. We also propose the "Polish Ratio" method, an innovative measure of the degree of modification made by ChatGPT compared to the original human-written text.
  arXiv  Detail & Related papers  (2023-07-21T06:38:37Z)
• CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts [10.034193809833372]
  Malicious users could synthesize dummy academic content through ChatGPT. We present a large-scale CHatGPT-writtEn AbsTract dataset (CHEAT) to support the development of detection algorithms.
  arXiv  Detail & Related papers  (2023-04-24T11:19:33Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Is ChatGPT A Good Keyphrase Generator? A Preliminary Study [51.863368917344864]
  ChatGPT has recently garnered significant attention from the computational linguistics community. We evaluate its performance in various aspects, including keyphrase generation prompts, keyphrase generation diversity, and long document understanding. We find that ChatGPT performs exceptionally well on all six candidate prompts, with minor performance differences observed across the datasets.
  arXiv  Detail & Related papers  (2023-03-23T02:50:38Z)
• ChatGPT as the Transportation Equity Information Source for Scientific Writing [0.0]
  This study explored the content and usefulness of ChatGPT-generated information related to transportation equity. It utilized 152 papers retrieved through the Web of Science (WoS) repository. The results indicate that a weak similarity between ChatGPT and human-written abstracts.
  arXiv  Detail & Related papers  (2023-03-10T16:21:54Z)
• Is ChatGPT a Good NLG Evaluator? A Preliminary Study [121.77986688862302]
  We provide a preliminary meta-evaluation on ChatGPT to show its reliability as an NLG metric. Experimental results show that compared with previous automatic metrics, ChatGPT achieves state-of-the-art or competitive correlation with human judgments. We hope our preliminary study could prompt the emergence of a general-purposed reliable NLG metric.
  arXiv  Detail & Related papers  (2023-03-07T16:57:20Z)
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver? [113.22611481694825]
  Large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community. It is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot.
  arXiv  Detail & Related papers  (2023-02-08T09:44:51Z)
• A Categorical Archive of ChatGPT Failures [47.64219291655723]
  ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study.
  arXiv  Detail & Related papers  (2023-02-06T04:21:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.