Automatic essay scoring: leveraging Jaccard coefficient and Cosine similaritywith n-gram variation in vector space model approach

• URL: http://arxiv.org/abs/2510.15311v1
• Date: Fri, 17 Oct 2025 04:54:12 GMT
• Title: Automatic essay scoring: leveraging Jaccard coefficient and Cosine similaritywith n-gram variation in vector space model approach
• Authors: Andharini Dwi Cahyani, Moh. Wildan Fathoni, Fika Hastarita Rachman, Ari Basuki, Salman Amin, Bain Khusnul Khotimah,
• Abstract summary: This study investigates the effectiveness of two popular similarity metrics, Jaccard coefficient, and Cosine similarity.<n>The performance of the system is evaluated by analyzing the root mean square error (RMSE), which measures the difference between the scores given by human graders and those generated by the system.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Automated essay scoring (AES) is a vital area of research aiming to provide efficient and accurate assessment tools for evaluating written content. This study investigates the effectiveness of two popular similarity metrics, Jaccard coefficient, and Cosine similarity, within the context of vector space models(VSM)employing unigram, bigram, and trigram representations. The data used in this research was obtained from the formative essay of the citizenship education subject in a junior high school. Each essay undergoes preprocessing to extract features using n-gram models, followed by vectorization to transform text data into numerical representations. Then, similarity scores are computed between essays using both Jaccard coefficient and Cosine similarity. The performance of the system is evaluated by analyzing the root mean square error (RMSE), which measures the difference between the scores given by human graders and those generated by the system. The result shows that the Cosine similarity outperformed the Jaccard coefficient. In terms of n-gram, unigrams have lower RMSE compared to bigrams and trigrams.

Related papers

• Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping [85.48043537327258]
  Contextual Dynamic Mapping (CDM) is a novel cross-tokenizer distillation framework.<n>It uses contextual information to enhance sequence alignment precision and dynamically improve vocabulary mapping.<n>Our method shows significant advantages over existing cross-tokenizer distillation baselines across diverse benchmarks.
  arXiv  Detail & Related papers  (2025-02-16T12:46:07Z)
• A Top-down Graph-based Tool for Modeling Classical Semantic Maps: A Crosslinguistic Case Study of Supplementary Adverbs [50.982315553104975]
  Semantic map models (SMMs) construct a network-like conceptual space from cross-linguistic instances or forms.<n>Most SMMs are manually built by human experts using bottom-up procedures.<n>We propose a novel graph-based algorithm that automatically generates conceptual spaces and SMMs in a top-down manner.
  arXiv  Detail & Related papers  (2024-12-02T12:06:41Z)
• Differentiable Optimization of Similarity Scores Between Models and Brains [1.5391321019692434]
  Similarity measures such as linear regression, Centered Kernel Alignment (CKA), Normalized Bures Similarity (NBS), and angular Procrustes distance are often used to quantify this similarity.<n>Here, we introduce a novel tool to investigate what drives high similarity scores and what constitutes a "good" score.<n>Surprisingly, we find that high similarity scores do not guarantee encoding task-relevant information in a manner consistent with neural data.
  arXiv  Detail & Related papers  (2024-07-09T17:31:47Z)
• Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores [28.431348662950743]
  We release Python package for measuring and extracting repetition in text.<n>We build a platform based on diversity for users to interactively explore repetition in text.
  arXiv  Detail & Related papers  (2024-03-01T14:23:12Z)
• Cobra Effect in Reference-Free Image Captioning Metrics [58.438648377314436]
  A proliferation of reference-free methods, leveraging visual-language pre-trained models (VLMs), has emerged. In this paper, we study if there are any deficiencies in reference-free metrics. We employ GPT-4V as an evaluative tool to assess generated sentences and the result reveals that our approach achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2024-02-18T12:36:23Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Tragic and Comical Networks. Clustering Dramatic Genres According to Structural Properties [0.0]
  A growing tradition in the joint field of network studies and drama history produces interpretations from the character networks of the plays. Our aim is to create a method that is able to cluster texts with similar structures on the basis of the play's well-interpretable and simple properties. Finding these features is the most important part of our research, as well as establishing the appropriate statistical procedure to calculate the similarities between the texts.
  arXiv  Detail & Related papers  (2023-02-16T12:36:16Z)
• SMART: Sentences as Basic Units for Text Evaluation [48.5999587529085]
  In this paper, we introduce a new metric called SMART to mitigate such limitations. We treat sentences as basic units of matching instead of tokens, and use a sentence matching function to soft-match candidate and reference sentences. Our results show that system-level correlations of our proposed metric with a model-based matching function outperforms all competing metrics.
  arXiv  Detail & Related papers  (2022-08-01T17:58:05Z)
• Towards Similarity-Aware Time-Series Classification [51.2400839966489]
  We study time-series classification (TSC), a fundamental task of time-series data mining. We propose Similarity-Aware Time-Series Classification (SimTSC), a framework that models similarity information with graph neural networks (GNNs)
  arXiv  Detail & Related papers  (2022-01-05T02:14:57Z)
• Off-Line Arabic Handwritten Words Segmentation using Morphological Operators [0.0]
  The framework is proposed based on three steps: pre-processing, segmentation, and evaluation. The proposed model achieved the highest accuracy when compared with the related works.
  arXiv  Detail & Related papers  (2021-01-07T23:38:53Z)
• Comparative Analysis of N-gram Text Representation on Igbo Text Document Similarity [0.0]
  The improvement in Information Technology has encouraged the use of Igbo in the creation of text such as resources and news articles online. It adopted Euclidean similarity measure to determine the similarities between Igbo text documents represented with two word-based n-gram text representation (unigram and bigram) models.
  arXiv  Detail & Related papers  (2020-04-01T12:24:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.