Detecting Generated Scientific Papers using an Ensemble of Transformer Models

• URL: http://arxiv.org/abs/2209.08283v1
• Date: Sat, 17 Sep 2022 08:43:25 GMT
• Title: Detecting Generated Scientific Papers using an Ensemble of Transformer Models
• Authors: Anna Glazkova and Maksim Glazkov
• Abstract summary: The paper describes neural models developed for the DAGPap22 shared task hosted at the Third Workshop on Scholarly Document Processing. Our work focuses on comparing different transformer-based models as well as using additional datasets and techniques to deal with imbalanced classes.
• Score: 4.56877715768796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The paper describes neural models developed for the DAGPap22 shared task hosted at the Third Workshop on Scholarly Document Processing. This shared task targets the automatic detection of generated scientific papers. Our work focuses on comparing different transformer-based models as well as using additional datasets and techniques to deal with imbalanced classes. As a final submission, we utilized an ensemble of SciBERT, RoBERTa, and DeBERTa fine-tuned using random oversampling technique. Our model achieved 99.24% in terms of F1-score. The official evaluation results have put our system at the third place.

Related papers

• Stacking the Odds: Transformer-Based Ensemble for AI-Generated Text Detection [3.2047868962340327]
  We use a stacking ensemble of Transformers for the task of AI-generated text detection. We show that ensembling the models results in an improved accuracy in comparison with using them individually.
  arXiv  Detail & Related papers  (2023-10-29T05:28:44Z)
• Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data [75.20035991513564]
  We introduce 3S Testing, a deep generative modeling framework to facilitate model evaluation. Our experiments demonstrate that 3S Testing outperforms traditional baselines. These results raise the question of whether we need a paradigm shift away from limited real test data towards synthetic test data.
  arXiv  Detail & Related papers  (2023-10-25T10:18:44Z)
• An Empirical Study of Multimodal Model Merging [148.48412442848795]
  Model merging is a technique that fuses multiple models trained on different tasks to generate a multi-task solution. We conduct our study for a novel goal where we can merge vision, language, and cross-modal transformers of a modality-specific architecture. We propose two metrics that assess the distance between weights to be merged and can serve as an indicator of the merging outcomes.
  arXiv  Detail & Related papers  (2023-04-28T15:43:21Z)
• RoBLEURT Submission for the WMT2021 Metrics Task [72.26898579202076]
  We present our submission to the Shared Metrics Task: RoBLEURT. Our model reaches state-of-the-art correlations with the WMT 2020 human annotations upon 8 out of 10 to-English language pairs.
  arXiv  Detail & Related papers  (2022-04-28T08:49:40Z)
• FH-SWF SG at GermEval 2021: Using Transformer-Based Language Models to Identify Toxic, Engaging, & Fact-Claiming Comments [0.0]
  We describe the methods we used for our submissions to the GermEval 2021 shared task on the identification of toxic, engaging, and fact-claiming comments. For all three subtasks we fine-tuned freely available transformer-based models from the Huggingface model hub. We evaluated the performance of various pre-trained models after fine-tuning on 80% of the training data and submitted predictions of the two best performing resulting models.
  arXiv  Detail & Related papers  (2021-09-07T09:46:27Z)
• Using Transformer based Ensemble Learning to classify Scientific Articles [0.0]
  It comprises four independent sub-systems capable of classifying abstracts of scientific literature to one of the given seven classes. We ensemble predictions of these four sub-systems using majority voting to develop the final system which gives a F1 score of 0.93 on the test and validation set.
  arXiv  Detail & Related papers  (2021-02-19T15:42:26Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)
• syrapropa at SemEval-2020 Task 11: BERT-based Models Design For Propagandistic Technique and Span Detection [2.0051855303186046]
  We first build the model for Span Identification (SI) based on SpanBERT, and facilitate the detection by a deeper model and a sentence-level representation. We then develop a hybrid model for the Technique Classification (TC) The hybrid model is composed of three submodels including two BERT models with different training methods, and a feature-based Logistic Regression model.
  arXiv  Detail & Related papers  (2020-08-24T02:15:29Z)
• From Sound Representation to Model Robustness [82.21746840893658]
  We investigate the impact of different standard environmental sound representations (spectrograms) on the recognition performance and adversarial attack robustness of a victim residual convolutional neural network. Averaged over various experiments on three environmental sound datasets, we found the ResNet-18 model outperforms other deep learning architectures.
  arXiv  Detail & Related papers  (2020-07-27T17:30:49Z)
• Gradient-Based Adversarial Training on Transformer Networks for Detecting Check-Worthy Factual Claims [3.7543966923106438]
  We introduce the first adversarially-regularized, transformer-based claim spotter model. We obtain a 4.70 point F1-score improvement over current state-of-the-art models. We propose a method to apply adversarial training to transformer models.
  arXiv  Detail & Related papers  (2020-02-18T16:51:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.