Classification of Human- and AI-Generated Texts for English, French, German, and Spanish

• URL: http://arxiv.org/abs/2312.04882v1
• Date: Fri, 8 Dec 2023 07:42:06 GMT
• Title: Classification of Human- and AI-Generated Texts for English, French, German, and Spanish
• Authors: Kristina Schaaff, Tim Schlippe, Lorenz Mindner
• Abstract summary: We analyze features to classify human- and AI-generated text for English, French, German and Spanish. For the detection of AI-generated text, the combination of all proposed features performs best. For the detection of AI-rephrased text, the systems with all features outperform systems with other features in many cases.
• Score: 0.138120109831448
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we analyze features to classify human- and AI-generated text for English, French, German and Spanish and compare them across languages. We investigate two scenarios: (1) The detection of text generated by AI from scratch, and (2) the detection of text rephrased by AI. For training and testing the classifiers in this multilingual setting, we created a new text corpus covering 10 topics for each language. For the detection of AI-generated text, the combination of all proposed features performs best, indicating that our features are portable to other related languages: The F1-scores are close with 99% for Spanish, 98% for English, 97% for German and 95% for French. For the detection of AI-rephrased text, the systems with all features outperform systems with other features in many cases, but using only document features performs best for German (72%) and Spanish (86%) and only text vector features leads to best results for English (78%).

Related papers

• Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection [49.27067541740956]
  Speech Emotion Recognition (SER) is a crucial component in developing general-purpose AI agents capable of natural human-computer interaction. Building robust multilingual SER systems remains challenging due to the scarcity of labeled data in languages other than English and Chinese. We propose an approach to enhance SER performance in low SER resource languages by leveraging data from high-resource languages.
  arXiv  Detail & Related papers  (2024-09-17T08:36:45Z)
• Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text [61.22649031769564]
  We propose a novel framework, paraphrased text span detection (PTD) PTD aims to identify paraphrased text spans within a text. We construct a dedicated dataset, PASTED, for paraphrased text span detection.
  arXiv  Detail & Related papers  (2024-05-21T11:22:27Z)
• MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection Benchmark [10.92793962395538]
  MultiTuDE is a novel benchmarking dataset for multilingual machine-generated text detection. It consists of 74,081 authentic and machine-generated texts in 11 languages. We compare the performance of zero-shot (statistical and black-box) and fine-tuned detectors.
  arXiv  Detail & Related papers  (2023-10-20T15:57:17Z)
• Generative AI Text Classification using Ensemble LLM Approaches [0.12483023446237698]
  Large Language Models (LLMs) have shown impressive performance across a variety of AI and natural language processing tasks. We propose an ensemble neural model that generates probabilities from different pre-trained LLMs. For the first task of distinguishing between AI and human generated text, our model ranked in fifth and thirteenth place.
  arXiv  Detail & Related papers  (2023-09-14T14:41:46Z)
• Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT [0.25782420501870296]
  We explore traditional and new features to detect text generated by AI from scratch and text rephrased by AI. For our experiments, we produced a new text corpus covering 10 school topics. Our best systems for classifying basic and advanced human-generated/AI-rephrased texts have F1-scores of more than 78%.
  arXiv  Detail & Related papers  (2023-08-10T05:09:42Z)
• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense [56.077252790310176]
  We present a paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking. We introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider.
  arXiv  Detail & Related papers  (2023-03-23T16:29:27Z)
• From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition [50.93943755401025]
  We propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition. We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement. Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses.
  arXiv  Detail & Related papers  (2023-01-19T02:37:56Z)
• MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsing [48.216386761482525]
  We present MultiSpider, the largest multilingual text-to- schema- dataset which covers seven languages (English, German, French, Spanish, Japanese, Chinese, and Vietnamese) Experimental results under three typical settings (zero-shot, monolingual and multilingual) reveal a 6.1% absolute drop in accuracy in non-English languages. We also propose a simple framework augmentation framework SAVe (Augmentation-with-Verification) which boosts the overall performance by about 1.8% and closes the 29.5% performance gap across languages.
  arXiv  Detail & Related papers  (2022-12-27T13:58:30Z)
• RuArg-2022: Argument Mining Evaluation [69.87149207721035]
  This paper is a report of the organizers on the first competition of argumentation analysis systems dealing with Russian language texts. A corpus containing 9,550 sentences (comments on social media posts) on three topics related to the COVID-19 pandemic was prepared. The system that won the first place in both tasks used the NLI (Natural Language Inference) variant of the BERT architecture.
  arXiv  Detail & Related papers  (2022-06-18T17:13:37Z)
• MultiAzterTest: a Multilingual Analyzer on Multiple Levels of Language for Readability Assessment [0.0]
  MultiAzterTest is an open source NLP tool that analyzes texts on over 125 measures of cohesion,language, and readability for English, Spanish and Basque. Using cross-lingual features, MultiAzterTest also obtains competitive results above all in a complex vs simple distinction.
  arXiv  Detail & Related papers  (2021-09-10T13:34:52Z)
• Feature Selection on Noisy Twitter Short Text Messages for Language Identification [0.0]
  We apply different feature selection algorithms across various learning algorithms in order to analyze the effect of the algorithm. The methodology focuses on the word level language identification using a novel dataset of 6903 tweets extracted from Twitter.
  arXiv  Detail & Related papers  (2020-07-11T09:22:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.