Enhancing Essay Scoring with Adversarial Weights Perturbation and Metric-specific AttentionPooling

• URL: http://arxiv.org/abs/2401.05433v1
• Date: Sat, 6 Jan 2024 06:05:12 GMT
• Title: Enhancing Essay Scoring with Adversarial Weights Perturbation and Metric-specific AttentionPooling
• Authors: Jiaxin Huang, Xinyu Zhao, Chang Che, Qunwei Lin, Bo Liu
• Abstract summary: This study explores the application of BERT-related techniques to enhance the assessment of ELLs' writing proficiency. To address the specific needs of ELLs, we propose the use of DeBERTa, a state-of-the-art neural language model.
• Score: 18.182517741584707
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The objective of this study is to improve automated feedback tools designed for English Language Learners (ELLs) through the utilization of data science techniques encompassing machine learning, natural language processing, and educational data analytics. Automated essay scoring (AES) research has made strides in evaluating written essays, but it often overlooks the specific needs of English Language Learners (ELLs) in language development. This study explores the application of BERT-related techniques to enhance the assessment of ELLs' writing proficiency within AES. To address the specific needs of ELLs, we propose the use of DeBERTa, a state-of-the-art neural language model, for improving automated feedback tools. DeBERTa, pretrained on large text corpora using self-supervised learning, learns universal language representations adaptable to various natural language understanding tasks. The model incorporates several innovative techniques, including adversarial training through Adversarial Weights Perturbation (AWP) and Metric-specific AttentionPooling (6 kinds of AP) for each label in the competition. The primary focus of this research is to investigate the impact of hyperparameters, particularly the adversarial learning rate, on the performance of the model. By fine-tuning the hyperparameter tuning process, including the influence of 6AP and AWP, the resulting models can provide more accurate evaluations of language proficiency and support tailored learning tasks for ELLs. This work has the potential to significantly benefit ELLs by improving their English language proficiency and facilitating their educational journey.

Related papers

• A Survey of Small Language Models [104.80308007044634]
  Small Language Models (SLMs) have become increasingly important due to their efficiency and performance to perform various language tasks with minimal computational resources. We present a comprehensive survey on SLMs, focusing on their architectures, training techniques, and model compression techniques.
  arXiv  Detail & Related papers  (2024-10-25T23:52:28Z)
• Scalable Language Model with Generalized Continual Learning [58.700439919096155]
  The Joint Adaptive Re-ization (JARe) is integrated with Dynamic Task-related Knowledge Retrieval (DTKR) to enable adaptive adjustment of language models based on specific downstream tasks. Our method demonstrates state-of-the-art performance on diverse backbones and benchmarks, achieving effective continual learning in both full-set and few-shot scenarios with minimal forgetting.
  arXiv  Detail & Related papers  (2024-04-11T04:22:15Z)
• Application of GPT Language Models for Innovation in Activities in University Teaching [0.0]
  GPT (Generative Pre-trained Transformer) language models are an artificial intelligence and natural language processing technology. There is a growing interest in applying GPT language models to university teaching in various dimensions.
  arXiv  Detail & Related papers  (2024-03-15T14:31:52Z)
• Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models [24.784439330058095]
  This study investigates concerns related to neural models inadvertently retaining personal or sensitive data. A novel approach is introduced to achieve precise and selective forgetting within language models. Two innovative evaluation metrics are proposed: Sensitive Information Extraction Likelihood (S-EL) and Sensitive Information Memory Accuracy (S-MA)
  arXiv  Detail & Related papers  (2024-02-08T16:50:01Z)
• Empirical Study of Large Language Models as Automated Essay Scoring Tools in English Composition__Taking TOEFL Independent Writing Task for Example [25.220438332156114]
  This study aims to assess the capabilities and constraints of ChatGPT, a prominent representative of large language models. This study employs ChatGPT to conduct an automated evaluation of English essays, even with a small sample size.
  arXiv  Detail & Related papers  (2024-01-07T07:13:50Z)
• SINC: Self-Supervised In-Context Learning for Vision-Language Tasks [64.44336003123102]
  We propose a framework to enable in-context learning in large language models. A meta-model can learn on self-supervised prompts consisting of tailored demonstrations. Experiments show that SINC outperforms gradient-based methods in various vision-language tasks.
  arXiv  Detail & Related papers  (2023-07-15T08:33:08Z)
• Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations [63.19448893196642]
  We present a novel framework that harnesses the rich contextual information and semantic representations provided by large language models to analyze behavior graphs. By leveraging this capability, our framework enables personalized and accurate job recommendations for individual users.
  arXiv  Detail & Related papers  (2023-07-10T11:29:41Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• Context Matters: A Strategy to Pre-train Language Model for Science Education [4.053049694533914]
  BERT-based language models have shown significant superiority over traditional NLP models in various language-related tasks. The language used by students is different from the language in journals and Wikipedia, which are training sources of BERT. Our study confirms the effectiveness of continual pre-training on domain-specific data in the education domain.
  arXiv  Detail & Related papers  (2023-01-27T23:50:16Z)
• Sample Efficient Approaches for Idiomaticity Detection [6.481818246474555]
  This work explores sample efficient methods of idiomaticity detection. In particular, we study the impact of Pattern Exploit Training (PET), a few-shot method of classification, and BERTRAM, an efficient method of creating contextual embeddings. Our experiments show that whilePET improves performance on English, they are much less effective on Portuguese and Galician, leading to an overall performance about on par with vanilla mBERT.
  arXiv  Detail & Related papers  (2022-05-23T13:46:35Z)
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing [73.16475763422446]
  We propose a multilingual robustness evaluation platform for NLP tasks (TextFlint) It incorporates universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analysis. TextFlint generates complete analytical reports as well as targeted augmented data to address the shortcomings of the model's robustness.
  arXiv  Detail & Related papers  (2021-03-21T17:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.