Keystroke Dynamics Against Academic Dishonesty in the Age of LLMs

• URL: http://arxiv.org/abs/2406.15335v1
• Date: Fri, 21 Jun 2024 17:51:26 GMT
• Title: Keystroke Dynamics Against Academic Dishonesty in the Age of LLMs
• Authors: Debnath Kundu, Atharva Mehta, Rajesh Kumar, Naman Lal, Avinash Anand, Apoorv Singh, Rajiv Ratn Shah,
• Abstract summary: This study proposes a keystroke dynamics-based method to differentiate between bona fide and assisted writing. To facilitate this, a dataset was developed to capture the keystroke patterns of individuals engaged in writing tasks. The detector, trained using a modified TypeNet architecture, achieved accuracies ranging from 74.98% to 85.72% in condition-specific scenarios and from 52.24% to 80.54% in condition-agnostic scenarios.
• Score: 25.683026758476835
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The transition to online examinations and assignments raises significant concerns about academic integrity. Traditional plagiarism detection systems often struggle to identify instances of intelligent cheating, particularly when students utilize advanced generative AI tools to craft their responses. This study proposes a keystroke dynamics-based method to differentiate between bona fide and assisted writing within academic contexts. To facilitate this, a dataset was developed to capture the keystroke patterns of individuals engaged in writing tasks, both with and without the assistance of generative AI. The detector, trained using a modified TypeNet architecture, achieved accuracies ranging from 74.98% to 85.72% in condition-specific scenarios and from 52.24% to 80.54% in condition-agnostic scenarios. The findings highlight significant differences in keystroke dynamics between genuine and assisted writing. The outcomes of this study enhance our understanding of how users interact with generative AI and have implications for improving the reliability of digital educational platforms.

Related papers

• Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection [29.433764586753956]
  Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks. The utilization of these models carries inherent risks, including but not limited to plagiarism, the dissemination of fake news, and issues in educational exercises. This paper aims to bridge this gap by constructing AIG-ASAP, an AI-generated student essay dataset.
  arXiv  Detail & Related papers  (2024-02-01T08:11:56Z)
• Generative AI in Writing Research Papers: A New Type of Algorithmic Bias and Uncertainty in Scholarly Work [0.38850145898707145]
  Large language models (LLMs) and generative AI tools present challenges in identifying and addressing biases. generative AI tools are susceptible to goal misgeneralization, hallucinations, and adversarial attacks such as red teaming prompts. We find that incorporating generative AI in the process of writing research manuscripts introduces a new type of context-induced algorithmic bias.
  arXiv  Detail & Related papers  (2023-12-04T04:05:04Z)
• Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models [56.257840490146]
  ConCue is a novel approach for improving visual feature extraction in HOI detection. We develop a transformer-based feature extraction module with a multi-tower architecture that integrates contextual cues into both instance and interaction detectors.
  arXiv  Detail & Related papers  (2023-11-26T09:11:32Z)
• Analysis of the User Perception of Chatbots in Education Using A Partial Least Squares Structural Equation Modeling Approach [0.0]
  Key behavior-related aspects, such as Optimism, Innovativeness, Discomfort, Insecurity, Transparency, Ethics, Interaction, Engagement, and Accuracy, were studied. Results showed that Optimism and Innovativeness are positively associated with Perceived Ease of Use (PEOU) and Perceived Usefulness (PU)
  arXiv  Detail & Related papers  (2023-11-07T00:44:56Z)
• HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis [13.098764928946208]
  HowkGPT is built upon a dataset of academic assignments and accompanying metadata. It computes perplexity scores for student-authored and ChatGPT-generated responses. It further refines its analysis by defining category-specific thresholds.
  arXiv  Detail & Related papers  (2023-05-26T11:07:25Z)
• Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach [77.61131357420201]
  We conduct a thorough and rigorous study on fairness disparities in peer review with the help of large language models (LMs) We collect, assemble, and maintain a comprehensive relational database for the International Conference on Learning Representations (ICLR) conference from 2017 to date. We postulate and study fairness disparities on multiple protective attributes of interest, including author gender, geography, author, and institutional prestige.
  arXiv  Detail & Related papers  (2022-11-07T16:19:42Z)
• CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation [91.16551253297588]
  COunterfactual Generation via Retrieval and Editing (CORE) is a retrieval-augmented generation framework for creating diverse counterfactual perturbations for training. CORE first performs a dense retrieval over a task-related unlabeled text corpus using a learned bi-encoder. CORE then incorporates these into prompts to a large language model with few-shot learning capabilities, for counterfactual editing.
  arXiv  Detail & Related papers  (2022-10-10T17:45:38Z)
• Toward Educator-focused Automated Scoring Systems for Reading and Writing [0.0]
  This paper addresses the challenges of data and label availability, authentic and extended writing, domain scoring, prompt and source variety, and transfer learning. It employs techniques that preserve essay length as an important feature without increasing model training costs.
  arXiv  Detail & Related papers  (2021-12-22T15:44:30Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems [64.4896118325552]
  We evaluate the current state-of-the-art AES models using a model adversarial evaluation scheme and associated metrics. We find that AES models are highly overstable. Even heavy modifications(as much as 25%) with content unrelated to the topic of the questions do not decrease the score produced by the models.
  arXiv  Detail & Related papers  (2020-07-14T03:49:43Z)
• Temporal Embeddings and Transformer Models for Narrative Text Understanding [72.88083067388155]
  We present two approaches to narrative text understanding for character relationship modelling. The temporal evolution of these relations is described by dynamic word embeddings, that are designed to learn semantic changes over time. A supervised learning approach based on the state-of-the-art transformer model BERT is used instead to detect static relations between characters.
  arXiv  Detail & Related papers  (2020-03-19T14:23:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.