Adaptation of the Multi-Concept Multivariate Elo Rating System to Medical Students Training Data

• URL: http://arxiv.org/abs/2403.07908v1
• Date: Mon, 26 Feb 2024 19:19:56 GMT
• Title: Adaptation of the Multi-Concept Multivariate Elo Rating System to Medical Students Training Data
• Authors: Erva Nihan Kandemir, Jill-Jenn Vie, Adam Sanchez-Ayte, Olivier Palombi, Franck Ramus,
• Abstract summary: Elo rating system is widely recognized for its proficiency in predicting student performance. This paper presents an adaptation of a multi concept variant of the Elo rating system to the data collected by a medical training platform.
• Score: 6.222836318380985
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate estimation of question difficulty and prediction of student performance play key roles in optimizing educational instruction and enhancing learning outcomes within digital learning platforms. The Elo rating system is widely recognized for its proficiency in predicting student performance by estimating both question difficulty and student ability while providing computational efficiency and real-time adaptivity. This paper presents an adaptation of a multi concept variant of the Elo rating system to the data collected by a medical training platform, a platform characterized by a vast knowledge corpus, substantial inter-concept overlap, a huge question bank with significant sparsity in user question interactions, and a highly diverse user population, presenting unique challenges. Our study is driven by two primary objectives: firstly, to comprehensively evaluate the Elo rating systems capabilities on this real-life data, and secondly, to tackle the issue of imprecise early stage estimations when implementing the Elo rating system for online assessments. Our findings suggest that the Elo rating system exhibits comparable accuracy to the well-established logistic regression model in predicting final exam outcomes for users within our digital platform. Furthermore, results underscore that initializing Elo rating estimates with historical data remarkably reduces errors and enhances prediction accuracy, especially during the initial phases of student interactions.

Related papers

• Rethinking Resource Management in Edge Learning: A Joint Pre-training and Fine-tuning Design Paradigm [87.47506806135746]
  In some applications, edge learning is experiencing a shift in focusing from conventional learning from scratch to new two-stage learning. This paper considers the problem of joint communication and computation resource management in a two-stage edge learning system. It is shown that the proposed joint resource management over the pre-training and fine-tuning stages well balances the system performance trade-off.
  arXiv  Detail & Related papers  (2024-04-01T00:21:11Z)
• Learning Fair Ranking Policies via Differentiable Optimization of Ordered Weighted Averages [55.04219793298687]
  This paper shows how efficiently-solvable fair ranking models can be integrated into the training loop of Learning to Rank. In particular, this paper is the first to show how to backpropagate through constrained optimizations of OWA objectives, enabling their use in integrated prediction and decision models.
  arXiv  Detail & Related papers  (2024-02-07T20:53:53Z)
• Admission Prediction in Undergraduate Applications: an Interpretable Deep Learning Approach [0.6906005491572401]
  This article addresses the challenge of validating the admission committee's decisions for undergraduate admissions. We propose deep learning-based classifiers, namely Feed-Forward and Input Convex neural networks. Our models achieve higher accuracy compared to the best-performing traditional machine learning-based approach by a considerable margin of 3.03%.
  arXiv  Detail & Related papers  (2024-01-22T05:44:43Z)
• Style Over Substance: Evaluation Biases for Large Language Models [17.13064447978519]
  This study investigates the behavior of crowd-sourced and expert annotators, as well as large language models (LLMs) Our findings reveal a concerning bias in the evaluation process, as answers with factual errors are rated more favorably than answers that are too short or contained grammatical errors. We propose independently evaluating machine-generated text across multiple dimensions, rather than merging all the evaluation aspects into a single score.
  arXiv  Detail & Related papers  (2023-07-06T14:42:01Z)
• A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems [128.63953314853327]
  "Lifelong Learning" systems are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. We show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems.
  arXiv  Detail & Related papers  (2023-01-18T21:58:54Z)
• Systematic Evaluation of Predictive Fairness [60.0947291284978]
  Mitigating bias in training on biased datasets is an important open problem. We examine the performance of various debiasing methods across multiple tasks. We find that data conditions have a strong influence on relative model performance.
  arXiv  Detail & Related papers  (2022-10-17T05:40:13Z)
• A Multicriteria Evaluation for Data-Driven Programming Feedback Systems: Accuracy, Effectiveness, Fallibility, and Students' Response [7.167352606079407]
  Data-driven programming feedback systems can help novices to program in the absence of a human tutor. Prior evaluations showed that these systems improve learning in terms of test scores, or task completion efficiency. These aspects include inherent fallibility of current state-of-the-art, students' programming behavior in response to correct/incorrect feedback, and effective/ineffective system components.
  arXiv  Detail & Related papers  (2022-07-27T00:29:32Z)
• Abnormal-aware Multi-person Evaluation System with Improved Fuzzy Weighting [0.0]
  We choose the two-stage screening method, which consists of rough screening and score-weighted Kendall-$tau$ Distance. We use Fuzzy Synthetic Evaluation Method(FSE) to determine the significance of scores given by reviewers as well as their reliability.
  arXiv  Detail & Related papers  (2022-05-01T03:42:43Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)
• Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification [75.49600684537117]
  Data management research is showing an increasing presence and interest in topics related to data and algorithmic fairness. We contribute a broad analysis of 13 fair classification approaches and additional variants, over their correctness, fairness, efficiency, scalability, and stability. Our analysis highlights novel insights on the impact of different metrics and high-level approach characteristics on different aspects of performance.
  arXiv  Detail & Related papers  (2021-01-18T22:55:40Z)
• ELMV: an Ensemble-Learning Approach for Analyzing Electrical Health Records with Significant Missing Values [4.9810955364960385]
  We propose a novel Ensemble-Learning for Missing Value (ELMV) framework, which introduces an effective approach to construct multiple subsets of the original EHR data with a much lower missing rate. ELMV has been evaluated on a real-world healthcare data for critical feature identification as well as a batch of simulation data with different missing rates for outcome prediction.
  arXiv  Detail & Related papers  (2020-06-25T06:29:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.