Predicting Desirable Revisions of Evidence and Reasoning in Argumentative Writing

• URL: http://arxiv.org/abs/2302.05039v1
• Date: Fri, 10 Feb 2023 03:59:59 GMT
• Title: Predicting Desirable Revisions of Evidence and Reasoning in Argumentative Writing
• Authors: Tazin Afrin and Diane Litman
• Abstract summary: We develop models to classify desirable evidence and desirable reasoning revisions in student argumentative writing. We explore two ways to improve performance - using the essay context of the revision, and using the feedback students received before the revision. Our results show that while a model using feedback information improves over a baseline model, models utilizing context - either alone or with feedback - are the most successful in identifying desirable revisions.
• Score: 1.0878040851638
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop models to classify desirable evidence and desirable reasoning revisions in student argumentative writing. We explore two ways to improve classifier performance - using the essay context of the revision, and using the feedback students received before the revision. We perform both intrinsic and extrinsic evaluation for each of our models and report a qualitative analysis. Our results show that while a model using feedback information improves over a baseline model, models utilizing context - either alone or with feedback - are the most successful in identifying desirable revisions.

Related papers

• Enhancing Student Feedback Using Predictive Models in Visual Literacy Courses [2.366162376710038]
  This study uses Na"ive Bayes modeling to analyze peer review data obtained from an undergraduate visual literacy course over five years. Our findings highlight the utility of Na"ive Bayes modeling, particularly in the analysis of student comments based on parts of speech.
  arXiv  Detail & Related papers  (2024-05-23T20:02:36Z)
• CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation [87.44350003888646]
  Eval-Instruct can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting. CritiqueLLM is empirically shown to outperform ChatGPT and all the open-source baselines.
  arXiv  Detail & Related papers  (2023-11-30T16:52:42Z)
• A Comprehensive Evaluation and Analysis Study for Chinese Spelling Check [53.152011258252315]
  We show that using phonetic and graphic information reasonably is effective for Chinese Spelling Check. Models are sensitive to the error distribution of the test set, which reflects the shortcomings of models. The commonly used benchmark, SIGHAN, can not reliably evaluate models' performance.
  arXiv  Detail & Related papers  (2023-07-25T17:02:38Z)
• To Revise or Not to Revise: Learning to Detect Improvable Claims for Argumentative Writing Support [20.905660642919052]
  We explore the main challenges to identifying argumentative claims in need of specific revisions. We propose a new sampling strategy based on revision distance. We provide evidence that using contextual information and domain knowledge can further improve prediction results.
  arXiv  Detail & Related papers  (2023-05-26T10:19:54Z)
• Towards Personalized Review Summarization by Modeling Historical Reviews from Customer and Product Separately [59.61932899841944]
  Review summarization is a non-trivial task that aims to summarize the main idea of the product review in the E-commerce website. We propose the Heterogeneous Historical Review aware Review Summarization Model (HHRRS) We employ a multi-task framework that conducts the review sentiment classification and summarization jointly.
  arXiv  Detail & Related papers  (2023-01-27T12:32:55Z)
• Analyzing and Evaluating Faithfulness in Dialogue Summarization [67.07947198421421]
  We first perform the fine-grained human analysis on the faithfulness of dialogue summaries and observe that over 35% of generated summaries are faithfully inconsistent respective the source dialogues. We present a new model-level faithfulness evaluation method. It examines generation models with multi-choice questions created by rule-based transformations.
  arXiv  Detail & Related papers  (2022-10-21T07:22:43Z)
• Aspect-Controllable Opinion Summarization [58.5308638148329]
  We propose an approach that allows the generation of customized summaries based on aspect queries. Using a review corpus, we create a synthetic training dataset of (review, summary) pairs enriched with aspect controllers. We fine-tune a pretrained model using our synthetic dataset and generate aspect-specific summaries by modifying the aspect controllers.
  arXiv  Detail & Related papers  (2021-09-07T16:09:17Z)
• Dialogue Summarization with Supporting Utterance Flow Modeling and Fact Regularization [58.965859508695225]
  We propose an end-to-end neural model for dialogue summarization with two novel modules. The supporting utterance flow modeling helps to generate a coherent summary by smoothly shifting the focus from the former utterances to the later ones. The fact regularization encourages the generated summary to be factually consistent with the ground-truth summary during model training.
  arXiv  Detail & Related papers  (2021-08-03T03:09:25Z)
• Annotation and Classification of Evidence and Reasoning Revisions in Argumentative Writing [0.9449650062296824]
  We introduce an annotation scheme to capture the nature of sentence-level revisions of evidence use and reasoning. We show that reliable manual annotation can be achieved and that revision annotations correlate with a holistic assessment of essay improvement.
  arXiv  Detail & Related papers  (2021-07-14T20:58:26Z)
• How to Evaluate a Summarizer: Study Design and Statistical Analysis for Manual Linguistic Quality Evaluation [3.624563211765782]
  We show that best choice of evaluation method can vary from one aspect to another. We show that the total number of annotators can have a strong impact on study power. Current statistical analysis methods can inflate type I error rates up to eight-fold.
  arXiv  Detail & Related papers  (2021-01-27T10:14:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.