An Empirical Study of AI Generated Text Detection Tools
- URL: http://arxiv.org/abs/2310.01423v1
- Date: Wed, 27 Sep 2023 12:44:12 GMT
- Title: An Empirical Study of AI Generated Text Detection Tools
- Authors: Arslan Akram
- Abstract summary: ChatGPT has great promise, but there are serious problems that might arise from its misuse.
Six different artificial intelligence (AI) text identification systems, including "GPTkit," "GPTZero," "Originality," "Sapling," "Writer," and "Zylalab," have accuracy rates between 55.29 and 97.0%.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Since ChatGPT has emerged as a major AIGC model, providing high-quality
responses across a wide range of applications (including software development
and maintenance), it has attracted much interest from many individuals. ChatGPT
has great promise, but there are serious problems that might arise from its
misuse, especially in the realms of education and public safety. Several AIGC
detectors are available, and they have all been tested on genuine text.
However, more study is needed to see how effective they are for multi-domain
ChatGPT material. This study aims to fill this need by creating a multi-domain
dataset for testing the state-of-the-art APIs and tools for detecting
artificially generated information used by universities and other research
institutions. A large dataset consisting of articles, abstracts, stories, news,
and product reviews was created for this study. The second step is to use the
newly created dataset to put six tools through their paces. Six different
artificial intelligence (AI) text identification systems, including "GPTkit,"
"GPTZero," "Originality," "Sapling," "Writer," and "Zylalab," have accuracy
rates between 55.29 and 97.0%. Although all the tools fared well in the
evaluations, originality was particularly effective across the board.
Related papers
- Artificial Intelligence Driven Course Generation: A Case Study Using ChatGPT [0.0]
The study aims to elaborate on using ChatGPT to create course materials.
The main objective is to assess the efficiency, quality, and impact of AI-driven course generation.
The study highlights the potential of AI to revolutionize educational content creation.
arXiv Detail & Related papers (2024-11-02T21:59:02Z) - SONAR: A Synthetic AI-Audio Detection Framework and Benchmark [59.09338266364506]
SONAR is a synthetic AI-Audio Detection Framework and Benchmark.
It aims to provide a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content.
It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based deepfake detection systems.
arXiv Detail & Related papers (2024-10-06T01:03:42Z) - Quantitative Analysis of AI-Generated Texts in Academic Research: A Study of AI Presence in Arxiv Submissions using AI Detection Tool [0.0]
This study will analyze a method that can see purposely manufactured content that academic organizations use to post on Arxiv.
The statistical analysis shows that Originality.ai is very accurate, with a rate of 98%.
arXiv Detail & Related papers (2024-02-09T17:20:48Z) - Towards Possibilities & Impossibilities of AI-generated Text Detection:
A Survey [97.33926242130732]
Large Language Models (LLMs) have revolutionized the domain of natural language processing (NLP) with remarkable capabilities of generating human-like text responses.
Despite these advancements, several works in the existing literature have raised serious concerns about the potential misuse of LLMs.
To address these concerns, a consensus among the research community is to develop algorithmic solutions to detect AI-generated text.
arXiv Detail & Related papers (2023-10-23T18:11:32Z) - ConDA: Contrastive Domain Adaptation for AI-generated Text Detection [17.8787054992985]
Large language models (LLMs) are increasingly being used for generating text in news articles.
Given the potential malicious nature in which these LLMs can be used to generate disinformation at scale, it is important to build effective detectors for such AI-generated text.
In this work we tackle this data problem, in detecting AI-generated news text, and frame the problem as an unsupervised domain adaptation task.
arXiv Detail & Related papers (2023-09-07T19:51:30Z) - FacTool: Factuality Detection in Generative AI -- A Tool Augmented
Framework for Multi-Task and Multi-Domain Scenarios [87.12753459582116]
A wider range of tasks now face an increasing risk of containing factual errors when handled by generative models.
We propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models.
arXiv Detail & Related papers (2023-07-25T14:20:51Z) - Testing of Detection Tools for AI-Generated Text [0.0]
The paper examines the functionality of detection tools for artificial intelligence generated text.
It evaluates them based on accuracy and error type analysis.
The research covers 12 publicly available tools and two commercial systems.
arXiv Detail & Related papers (2023-06-21T16:29:44Z) - M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box
Machine-Generated Text Detection [69.29017069438228]
Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries.
This has also raised concerns about the potential misuse of such texts in journalism, education, and academia.
In this study, we strive to create automated systems that can detect machine-generated texts and pinpoint potential misuse.
arXiv Detail & Related papers (2023-05-24T08:55:11Z) - On the Possibilities of AI-Generated Text Detection [76.55825911221434]
We argue that as machine-generated text approximates human-like quality, the sample size needed for detection bounds increases.
We test various state-of-the-art text generators, including GPT-2, GPT-3.5-Turbo, Llama, Llama-2-13B-Chat-HF, and Llama-2-70B-Chat-HF, against detectors, including oBERTa-Large/Base-Detector, GPTZero.
arXiv Detail & Related papers (2023-04-10T17:47:39Z) - To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection.
We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains.
Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
arXiv Detail & Related papers (2023-04-04T03:04:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.