SemEval-2024 Task 8: Weighted Layer Averaging RoBERTa for Black-Box Machine-Generated Text Detection

• URL: http://arxiv.org/abs/2402.15873v2
• Date: Tue, 9 Apr 2024 10:19:48 GMT
• Title: SemEval-2024 Task 8: Weighted Layer Averaging RoBERTa for Black-Box Machine-Generated Text Detection
• Authors: Ayan Datta, Aryan Chandramania, Radhika Mamidi,
• Abstract summary: This document includes the details of the authors' submission to the proceedings of SemEval 2024's Task 8: Multigenerator, Multidomain, and Multilingual Black-Box Machine-generated Text Detection Subtask A (monolingual) and B.
• Score: 5.049812996253857
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This document contains the details of the authors' submission to the proceedings of SemEval 2024's Task 8: Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection Subtask A (monolingual) and B. Detection of machine-generated text is becoming an increasingly important task, with the advent of large language models (LLMs). In this paper, we lay out how using weighted averages of RoBERTa layers lets us capture information about text that is relevant to machine-generated text detection.

Related papers

• GigaCheck: Detecting LLM-generated Content [72.27323884094953]
  In this work, we investigate the task of generated text detection by proposing the GigaCheck. Our research explores two approaches: (i) distinguishing human-written texts from LLM-generated ones, and (ii) detecting LLM-generated intervals in Human-Machine collaborative texts. Specifically, we use a fine-tuned general-purpose LLM in conjunction with a DETR-like detection model, adapted from computer vision, to localize AI-generated intervals within text.
  arXiv  Detail & Related papers  (2024-10-31T08:30:55Z)
• Mast Kalandar at SemEval-2024 Task 8: On the Trail of Textual Origins: RoBERTa-BiLSTM Approach to Detect AI-Generated Text [7.959800630494841]
  SemEval 2024 introduces the task of Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection. We propose a RoBERTa-BiLSTM based classifier designed to classify text into two categories: AI-generated or human. Our architecture ranked 46th on the official leaderboard with an accuracy of 80.83 among 125.
  arXiv  Detail & Related papers  (2024-07-03T10:22:23Z)
• SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection [68.858931667807]
  Subtask A is a binary classification task determining whether a text is written by a human or generated by a machine. Subtask B is to detect the exact source of a text, discerning whether it is written by a human or generated by a specific LLM. Subtask C aims to identify the changing point within a text, at which the authorship transitions from human to machine.
  arXiv  Detail & Related papers  (2024-04-22T13:56:07Z)
• PetKaz at SemEval-2024 Task 8: Can Linguistics Capture the Specifics of LLM-generated Text? [4.463184061618504]
  We present our submission to the SemEval-2024 Task 8 "Multigenerator, Multidomain, and Black-Box Machine-Generated Text Detection" Our approach relies on combining embeddings from the RoBERTa-base with diversity features and uses a resampled training set. Our results show that our approach is generalizable across unseen models and domains, achieving an accuracy of 0.91.
  arXiv  Detail & Related papers  (2024-04-08T13:05:02Z)
• TrustAI at SemEval-2024 Task 8: A Comprehensive Analysis of Multi-domain Machine Generated Text Detection Techniques [2.149586323955579]
  Large Language Models (LLMs) generate fluent content across a wide spectrum of user queries. This capability has raised concerns regarding misinformation and personal information leakage. We present our methods for the SemEval2024 Task8, aiming to detect machine-generated text across various domains.
  arXiv  Detail & Related papers  (2024-03-25T10:09:03Z)
• MasonTigers at SemEval-2024 Task 8: Performance Analysis of Transformer-based Models on Machine-Generated Text Detection [0.0]
  The task encompasses Binary Human-Written vs. Machine-Generated Text Classification (Track A), Multi-Way Machine-Generated Text Classification (Track B), and Human-Machine Mixed Text Detection (Track C) Our best performing approaches utilize mainly the ensemble of discriminator transformer models along with sentence transformer and statistical machine learning approaches in specific cases.
  arXiv  Detail & Related papers  (2024-03-22T06:47:28Z)
• Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs [96.54224331778195]
  We present a text-grounding document understanding model, termed TGDoc, which enhances MLLMs with the ability to discern the spatial positioning of text within images. We formulate instruction tuning tasks including text detection, recognition, and spotting to facilitate the cohesive alignment between the visual encoder and large language model. Our method achieves state-of-the-art performance across multiple text-rich benchmarks, validating the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-11-22T06:46:37Z)
• M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection [69.29017069438228]
  Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries. This has also raised concerns about the potential misuse of such texts in journalism, education, and academia. In this study, we strive to create automated systems that can detect machine-generated texts and pinpoint potential misuse.
  arXiv  Detail & Related papers  (2023-05-24T08:55:11Z)
• DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection [56.513637720967566]
  Large language models (LLMs) can generate texts that pose risks of misuse, such as plagiarism, planting fake reviews on e-commerce platforms, or creating inflammatory false tweets. Existing high-quality detection methods usually require access to the interior of the model to extract the intrinsic characteristics. We propose to extract deep intrinsic characteristics of the black-box model generated texts.
  arXiv  Detail & Related papers  (2023-05-21T17:26:16Z)
• On the Possibilities of AI-Generated Text Detection [76.55825911221434]
  We argue that as machine-generated text approximates human-like quality, the sample size needed for detection bounds increases. We test various state-of-the-art text generators, including GPT-2, GPT-3.5-Turbo, Llama, Llama-2-13B-Chat-HF, and Llama-2-70B-Chat-HF, against detectors, including oBERTa-Large/Base-Detector, GPTZero.
  arXiv  Detail & Related papers  (2023-04-10T17:47:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.