DIALOG-22 RuATD Generated Text Detection

• URL: http://arxiv.org/abs/2206.08029v1
• Date: Thu, 16 Jun 2022 09:33:26 GMT
• Title: DIALOG-22 RuATD Generated Text Detection
• Authors: Narek Maloyan, Bulat Nutfullin, Eugene Ilyushin
• Abstract summary: Detectors that can distinguish between TGM-generated text and human-written ones play an important role in preventing abuse of TGM. We describe our pipeline for the two DIALOG-22 RuATD tasks: detecting generated text (binary task) and classification of which model was used to generate text.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text Generation Models (TGMs) succeed in creating text that matches human language style reasonably well. Detectors that can distinguish between TGM-generated text and human-written ones play an important role in preventing abuse of TGM. In this paper, we describe our pipeline for the two DIALOG-22 RuATD tasks: detecting generated text (binary task) and classification of which model was used to generate text (multiclass task). We achieved 1st place on the binary classification task with an accuracy score of 0.82995 on the private test set and 4th place on the multiclass classification task with an accuracy score of 0.62856 on the private test set. We proposed an ensemble method of different pre-trained models based on the attention mechanism.

Related papers

• Sharif-MGTD at SemEval-2024 Task 8: A Transformer-Based Approach to Detect Machine Generated Text [2.2039952888743253]
  MGT has emerged as a significant area of study within Natural Language Processing. In this research, we explore the effectiveness of fine-tuning a RoBERTa-base transformer, a powerful neural architecture, to address MGT detection. Our proposed system achieves an accuracy of 78.9% on the test dataset, positioning us at 57th among participants.
  arXiv  Detail & Related papers  (2024-07-16T14:33:01Z)
• Mast Kalandar at SemEval-2024 Task 8: On the Trail of Textual Origins: RoBERTa-BiLSTM Approach to Detect AI-Generated Text [7.959800630494841]
  SemEval 2024 introduces the task of Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection. We propose a RoBERTa-BiLSTM based classifier designed to classify text into two categories: AI-generated or human. Our architecture ranked 46th on the official leaderboard with an accuracy of 80.83 among 125.
  arXiv  Detail & Related papers  (2024-07-03T10:22:23Z)
• Applying Ensemble Methods to Model-Agnostic Machine-Generated Text Detection [0.0]
  We study the problem of detecting machine-generated text when the large language model it is possibly derived from is unknown. We use a zero-shot model for machine-generated text detection which is highly accurate when the generative (or base) language model is the same as the discriminative (or scoring) language model.
  arXiv  Detail & Related papers  (2024-06-18T12:58:01Z)
• AISPACE at SemEval-2024 task 8: A Class-balanced Soft-voting System for Detecting Multi-generator Machine-generated Text [0.0]
  SemEval-2024 Task 8 provides a challenge to detect human-written and machine-generated text. This paper proposes a system that mainly deals with Subtask B. It aims to detect if given full text is written by human or is generated by a specific Large Language Model (LLM), which is actually a multi-class text classification task.
  arXiv  Detail & Related papers  (2024-04-01T06:25:47Z)
• DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection [56.513637720967566]
  Large language models (LLMs) can generate texts that pose risks of misuse, such as plagiarism, planting fake reviews on e-commerce platforms, or creating inflammatory false tweets. Existing high-quality detection methods usually require access to the interior of the model to extract the intrinsic characteristics. We propose to extract deep intrinsic characteristics of the black-box model generated texts.
  arXiv  Detail & Related papers  (2023-05-21T17:26:16Z)
• Smaller Language Models are Better Black-box Machine-Generated Text Detectors [56.36291277897995]
  Small and partially-trained models are better universal text detectors. We find that whether the detector and generator were trained on the same data is not critically important to the detection success. For instance, the OPT-125M model has an AUC of 0.81 in detecting ChatGPT generations, whereas a larger model from the GPT family, GPTJ-6B, has AUC of 0.45.
  arXiv  Detail & Related papers  (2023-05-17T00:09:08Z)
• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense [56.077252790310176]
  We present a paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Using DIPPER to paraphrase text generated by three large language models (including GPT3.5-davinci-003) successfully evades several detectors, including watermarking. We introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider.
  arXiv  Detail & Related papers  (2023-03-23T16:29:27Z)
• Code-Switching Text Generation and Injection in Mandarin-English ASR [57.57570417273262]
  We investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T) We first propose a strategy to generate code-switching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models.
  arXiv  Detail & Related papers  (2023-03-20T09:13:27Z)
• Classifiers are Better Experts for Controllable Text Generation [63.17266060165098]
  We show that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and sentiment accuracy based on the external classifier of generated texts. The same time, it is also easier to implement and tune, and has significantly fewer restrictions and requirements.
  arXiv  Detail & Related papers  (2022-05-15T12:58:35Z)
• UPB at SemEval-2021 Task 7: Adversarial Multi-Task Learning for Detecting and Rating Humor and Offense [0.6404122934568858]
  We describe our adversarial multi-task network, AMTL-Humor, used to detect and rate humor and offensive texts. Our best model consists of an ensemble of all tested configurations, and achieves a 95.66% F1-score and 94.70% accuracy for Task 1a.
  arXiv  Detail & Related papers  (2021-04-13T09:59:05Z)
• Improving Text Generation with Student-Forcing Optimal Transport [122.11881937642401]
  We propose using optimal transport (OT) to match the sequences generated in training and testing modes. An extension is also proposed to improve the OT learning, based on the structural and contextual information of the text sequences. The effectiveness of the proposed method is validated on machine translation, text summarization, and text generation tasks.
  arXiv  Detail & Related papers  (2020-10-12T19:42:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.