Overview of the Plagiarism Detection Task at PAN 2025

• URL: http://arxiv.org/abs/2510.06805v1
• Date: Wed, 08 Oct 2025 09:33:26 GMT
• Title: Overview of the Plagiarism Detection Task at PAN 2025
• Abstract summary: We created a novel large-scale dataset of automatically generated plagiarism using three large language models: Llama, DeepSeek-R1, and Mistral.<n>In this paper, we outline the creation of this dataset, summarize and compare the results of all participants and four baselines, and evaluate the results on the last plagiarism detection task from PAN 2015.
• Score: 41.899055581824314
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The generative plagiarism detection task at PAN 2025 aims at identifying automatically generated textual plagiarism in scientific articles and aligning them with their respective sources. We created a novel large-scale dataset of automatically generated plagiarism using three large language models: Llama, DeepSeek-R1, and Mistral. In this task overview paper, we outline the creation of this dataset, summarize and compare the results of all participants and four baselines, and evaluate the results on the last plagiarism detection task from PAN 2015 in order to interpret the robustness of the proposed approaches. We found that the current iteration does not invite a large variety of approaches as naive semantic similarity approaches based on embedding vectors provide promising results of up to 0.8 recall and 0.5 precision. In contrast, most of these approaches underperform significantly on the 2015 dataset, indicating a lack in generalizability.

Related papers

• Beyond Representational Similarity: Source-Conditioned Description-Length Gain for Generative Plagiarism Detection and Candidate Source Reranking [8.450813632047902]
  Large language models (LLMs) pose challenges to academic integrity and peer review.<n>generative plagiarism detection remains an underexplored and largely unresolved challenge.<n>Motivated by the description-length view of probabilistic prediction, we introduce Source-Conditioned Description-Length Gain (SCDG)<n>SCDG is a directional, training-free framework that contrasts a frozen language model's description length of a suspicious document $P$ with and without a candidate source $S$.
  arXiv  Detail & Related papers  (2026-08-04T16:03:45Z)
• Overview of PAN 2026: Voight-Kampff Generative AI Detection, Text Watermarking, Multi-Author Writing Style Analysis, Generative Plagiarism Detection, and Reasoning Trajectory Detection [58.873639022077235]
  The goal of the PAN workshop is to advance computational stylometry and text forensics via objective and reproducible evaluation.<n>As in previous years, PAN invites software submissions as easy-to-reproduce Docker containers for most of the tasks.<n>Since PAN 2012, more than 1,100 submissions have been made this way via the TIRA experimentation platform.
  arXiv  Detail & Related papers  (2026-02-09T19:48:16Z)
• AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research [81.04845910798387]
  Generating natural language explanations for threat detections remains an open problem in cybersecurity research.<n>We present AutoMalDesc, an automated static analysis summarization framework that operates independently at scale.<n>We publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) datasets, along with our methodology and evaluation framework.
  arXiv  Detail & Related papers  (2025-11-17T13:05:25Z)
• AgentExpt: Automating AI Experiment Design with LLM-based Resource Retrieval Agent [36.65355075707938]
  One key application in AI research is to automate experiment design through agentic and baseline retrieval.<n>We present a comprehensive framework for baseline and dataset recommendation.<n>We develop a reasoning-augmented reranker that exact interaction chains to construct explicit reasoning chains and finetunes a large language model to produce interpretable justifications.
  arXiv  Detail & Related papers  (2025-11-07T01:51:56Z)
• Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text [30.74255946385862]
  We introduce Text2Vis, a benchmark designed to assess text-to-visualization models.<n>It comprises 1,985 samples, each with a data table, natural language query, short answer, visualization code, and annotated charts.<n>It reveals significant performance gaps, highlighting key challenges, and offering insights for future advancements.
  arXiv  Detail & Related papers  (2025-07-26T14:59:04Z)
• Detecting Document-level Paraphrased Machine Generated Content: Mimicking Human Writing Style and Involving Discourse Features [57.34477506004105]
  Machine-generated content poses challenges such as academic plagiarism and the spread of misinformation.<n>We introduce novel methodologies and datasets to overcome these challenges.<n>We propose MhBART, an encoder-decoder model designed to emulate human writing style.<n>We also propose DTransformer, a model that integrates discourse analysis through PDTB preprocessing to encode structural features.
  arXiv  Detail & Related papers  (2024-12-17T08:47:41Z)
• BERT-Enhanced Retrieval Tool for Homework Plagiarism Detection System [0.0]
  We propose a plagiarized text data generation method based on GPT-3.5, which produces 32,927 pairs of text plagiarism detection datasets. We also propose a plagiarism identification method based on Faiss with BERT with high efficiency and high accuracy. Our experiments show that the performance of this model outperforms other models in several metrics, including 98.86%, 98.90%, 98.86%, and 0.9888 for Accuracy, Precision, Recall, and F1 Score.
  arXiv  Detail & Related papers  (2024-04-01T12:20:34Z)
• Contrastive Transformer Learning with Proximity Data Generation for Text-Based Person Search [60.626459715780605]
  Given a descriptive text query, text-based person search aims to retrieve the best-matched target person from an image gallery. Such a cross-modal retrieval task is quite challenging due to significant modality gap, fine-grained differences and insufficiency of annotated data. In this paper, we propose a simple yet effective dual Transformer model for text-based person search.
  arXiv  Detail & Related papers  (2023-11-15T16:26:49Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [49.15931834209624]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world.<n>We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique.<n>By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• Transferring BERT-like Transformers' Knowledge for Authorship Verification [8.443350618722562]
  We study the effectiveness of several BERT-like transformers for the task of authorship verification. We provide new splits for PAN-2020, where training and test data are sampled from disjoint topics or authors. We show that those splits can enhance the models' capability to transfer knowledge over a new, significantly different dataset.
  arXiv  Detail & Related papers  (2021-12-09T18:57:29Z)
• Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension [29.717816161964105]
  We propose a novel approach called Adaptive Bidirectional Attention, which adaptively exploits the source representations of different levels to the predictor. Results are better than the previous state-of-the-art model by 2.5$%$ EM and 2.3$%$ F1 scores.
  arXiv  Detail & Related papers  (2020-12-20T09:31:35Z)
• Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
  A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples. We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries. We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-02-27T10:22:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.