Is ChatGPT Transforming Academics' Writing Style?

• URL: http://arxiv.org/abs/2404.08627v1
• Date: Fri, 12 Apr 2024 17:41:05 GMT
• Title: Is ChatGPT Transforming Academics' Writing Style?
• Authors: Mingmeng Geng, Roberto Trotta,
• Abstract summary: Based on one million arXiv papers submitted from May 2018 to January 2024, we assess the textual density of ChatGPT's writing style in their abstracts. We find that ChatGPT is having an increasing impact on arXiv abstracts, especially in the field of computer science. We conclude with an analysis of both positive and negative aspects of the penetration of ChatGPT into academics' writing style.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Based on one million arXiv papers submitted from May 2018 to January 2024, we assess the textual density of ChatGPT's writing style in their abstracts by means of a statistical analysis of word frequency changes. Our model is calibrated and validated on a mixture of real abstracts and ChatGPT-modified abstracts (simulated data) after a careful noise analysis. We find that ChatGPT is having an increasing impact on arXiv abstracts, especially in the field of computer science, where the fraction of ChatGPT-revised abstracts is estimated to be approximately 35%, if we take the output of one of the simplest prompts, "revise the following sentences", as a baseline. We conclude with an analysis of both positive and negative aspects of the penetration of ChatGPT into academics' writing style.

Related papers

• Impact of ChatGPT on the writing style of condensed matter physicists [6.653378613306849]
  We estimate the impact of ChatGPT's release on the writing style of condensed matter papers on arXiv. Our analysis reveals a statistically significant improvement in the English quality of abstracts written by non-native English speakers.
  arXiv  Detail & Related papers  (2024-08-30T14:37:10Z)
• Can ChatGPT evaluate research quality? [3.9627148816681284]
  ChatGPT-4 can produce plausible document summaries and quality evaluation rationales that match REF criteria. Overall, ChatGPT does not yet seem to be accurate enough to be trusted for any formal or informal research quality evaluation tasks.
  arXiv  Detail & Related papers  (2024-02-08T10:00:40Z)
• Chatbot-supported Thesis Writing: An Autoethnographic Report [0.0]
  ChatGPT might be applied to formats that require learners to generate text, such as bachelor theses or student research papers. ChatGPT is to be valued as a beneficial tool in thesis writing. However, writing a conclusive thesis still requires the learner's meaningful engagement.
  arXiv  Detail & Related papers  (2023-10-14T09:09:26Z)
• Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text [48.36706154871577]
  We introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts) It diverges from extant corpora by comprising pairs of human-written and ChatGPT-polished abstracts instead of purely ChatGPT-generated texts. We also propose the "Polish Ratio" method, an innovative measure of the degree of modification made by ChatGPT compared to the original human-written text.
  arXiv  Detail & Related papers  (2023-07-21T06:38:37Z)
• On the Detectability of ChatGPT Content: Benchmarking, Methodology, and Evaluation through the Lens of Academic Writing [10.534162347659514]
  We develop a deep neural framework named CheckGPT to better capture the subtle and deep semantic and linguistic patterns in ChatGPT written literature. To evaluate the detectability of ChatGPT content, we conduct extensive experiments on the transferability, prompt engineering, and robustness of CheckGPT.
  arXiv  Detail & Related papers  (2023-06-07T12:33:24Z)
• Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study [51.079100495163736]
  This paper systematically inspects ChatGPT's performance in two discourse analysis tasks: topic segmentation and discourse parsing. ChatGPT demonstrates proficiency in identifying topic structures in general-domain conversations yet struggles considerably in specific-domain conversations. Our deeper investigation indicates that ChatGPT can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures.
  arXiv  Detail & Related papers  (2023-05-15T07:14:41Z)
• ChatLog: Carefully Evaluating the Evolution of ChatGPT Across Time [54.18651663847874]
  ChatGPT has achieved great success and can be considered to have acquired an infrastructural status. Existing benchmarks encounter two challenges: (1) Disregard for periodical evaluation and (2) Lack of fine-grained features. We construct ChatLog, an ever-updating dataset with large-scale records of diverse long-form ChatGPT responses for 21 NLP benchmarks from March, 2023 to now.
  arXiv  Detail & Related papers  (2023-04-27T11:33:48Z)
• CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts [10.034193809833372]
  Malicious users could synthesize dummy academic content through ChatGPT. We present a large-scale CHatGPT-writtEn AbsTract dataset (CHEAT) to support the development of detection algorithms.
  arXiv  Detail & Related papers  (2023-04-24T11:19:33Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Is ChatGPT a Good NLG Evaluator? A Preliminary Study [121.77986688862302]
  We provide a preliminary meta-evaluation on ChatGPT to show its reliability as an NLG metric. Experimental results show that compared with previous automatic metrics, ChatGPT achieves state-of-the-art or competitive correlation with human judgments. We hope our preliminary study could prompt the emergence of a general-purposed reliable NLG metric.
  arXiv  Detail & Related papers  (2023-03-07T16:57:20Z)
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver? [113.22611481694825]
  Large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community. It is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot.
  arXiv  Detail & Related papers  (2023-02-08T09:44:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.