Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring

• URL: http://arxiv.org/abs/2008.01809v1
• Date: Tue, 4 Aug 2020 20:13:51 GMT
• Title: Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring
• Authors: Haoran Zhang and Diane Litman
• Abstract summary: This paper presents a method for linking automated essay scoring (AES) and automated writing evaluation (AWE) We evaluate performance using a feature-based AES requiring Topical Components (TCs) Results show that performance is comparable whether using automatically or manually constructed TCs for 1) representing essays as rubric-based features, 2) grading essays.
• Score: 15.234595490118542
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While automated essay scoring (AES) can reliably grade essays at scale, automated writing evaluation (AWE) additionally provides formative feedback to guide essay revision. However, a neural AES typically does not provide useful feature representations for supporting AWE. This paper presents a method for linking AWE and neural AES, by extracting Topical Components (TCs) representing evidence from a source text using the intermediate output of attention layers. We evaluate performance using a feature-based AES requiring TCs. Results show that performance is comparable whether using automatically or manually constructed TCs for 1) representing essays as rubric-based features, 2) grading essays.

Related papers

• Transformer-based Joint Modelling for Automatic Essay Scoring and Off-Topic Detection [3.609048819576875]
  We are proposing an unsupervised technique that jointly scores essays and detects off-topic essays. Our proposed method outperforms the baseline we created and earlier conventional methods on two essay-scoring datasets.
  arXiv  Detail & Related papers  (2024-03-24T21:44:14Z)
• Prompt- and Trait Relation-aware Cross-prompt Essay Trait Scoring [3.6825890616838066]
  Automated essay scoring (AES) aims to score essays written for a given prompt, which defines the writing topic. Most existing AES systems assume to grade essays of the same prompt as used in training and assign only a holistic score. We propose a robust model: prompt- and trait relation-aware cross-prompt essay trait scorer.
  arXiv  Detail & Related papers  (2023-05-26T11:11:19Z)
• Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation [160.07938471250048]
  Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics. We develop strong-performing automatic metrics for reference-based summarization evaluation.
  arXiv  Detail & Related papers  (2023-03-07T02:49:50Z)
• Comparing Methods for Extractive Summarization of Call Centre Dialogue [77.34726150561087]
  We experimentally compare several such methods by using them to produce summaries of calls, and evaluating these summaries objectively. We found that TopicSum and Lead-N outperform the other summarisation methods, whilst BERTSum received comparatively lower scores in both subjective and objective evaluations.
  arXiv  Detail & Related papers  (2022-09-06T13:16:02Z)
• Deep Learning Architecture for Automatic Essay Scoring [0.0]
  We propose a novel architecture based on recurrent networks (RNN) and convolution neural network (CNN) In the proposed architecture, the multichannel convolutional layer learns and captures the contextual features of the word n-gram from the word embedding vectors. Our proposed system achieves significantly higher grading accuracy than other deep learning-based AES systems.
  arXiv  Detail & Related papers  (2022-06-16T14:56:24Z)
• Automated Evaluation for Student Argumentative Writing: A Survey [2.9466390764652415]
  This paper surveys and organizes research works in an under-studied area, which we call automated evaluation for student argumentative writing. Unlike traditional automated writing evaluation that focuses on holistic essay scoring, this field is more specific: it focuses on evaluating argumentative essays and offers specific feedback.
  arXiv  Detail & Related papers  (2022-05-09T07:27:59Z)
• Question-Based Salient Span Selection for More Controllable Text Summarization [67.68208237480646]
  We propose a method for incorporating question-answering (QA) signals into a summarization model. Our method identifies salient noun phrases (NPs) in the input document by automatically generating wh-questions that are answered by the NPs. This QA-based signal is incorporated into a two-stage summarization model which first marks salient NPs in the input document using a classification model, then conditionally generates a summary.
  arXiv  Detail & Related papers  (2021-11-15T17:36:41Z)
• Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays [41.851075178681015]
  We describe a way to score essays holistically using a multi-task learning (MTL) approach. We compare our results with a single-task learning (STL) approach, using both LSTMs and BiLSTMs. We find that MTL-based BiLSTM system gives the best results for scoring the essay holistically, as well as performing well on scoring the essay traits.
  arXiv  Detail & Related papers  (2021-02-01T11:31:09Z)
• Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding [71.2260967797055]
  We propose a weakly-supervised approach for aspect-based sentiment analysis. We learn sentiment, aspect> joint topic embeddings in the word embedding space. We then use neural models to generalize the word-level discriminative information.
  arXiv  Detail & Related papers  (2020-10-13T21:33:24Z)
• Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems [64.4896118325552]
  We evaluate the current state-of-the-art AES models using a model adversarial evaluation scheme and associated metrics. We find that AES models are highly overstable. Even heavy modifications(as much as 25%) with content unrelated to the topic of the questions do not decrease the score produced by the models.
  arXiv  Detail & Related papers  (2020-07-14T03:49:43Z)
• Improving Readability for Automatic Speech Recognition Transcription [50.86019112545596]
  We propose a novel NLP task called ASR post-processing for readability (APR) APR aims to transform the noisy ASR output into a readable text for humans and downstream tasks while maintaining the semantic meaning of the speaker. We compare fine-tuned models based on several open-sourced and adapted pre-trained models with the traditional pipeline method.
  arXiv  Detail & Related papers  (2020-04-09T09:26:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.