Diagnostic Assessment Generation via Combinatorial Search
- URL: http://arxiv.org/abs/2112.11188v1
- Date: Mon, 6 Dec 2021 06:19:15 GMT
- Title: Diagnostic Assessment Generation via Combinatorial Search
- Authors: Daehan Kim, Hyeonseong Choi, Guik Jung
- Abstract summary: We present a generic formulation of question assembly and a genetic based method that can generate assessment tests from raw problem-solving history.
Experimental results show that the proposed method outperforms greedy and random baseline by a large margin.
We also performed qualitative analysis on the generated assessment test for 9th graders.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Initial assessment tests are crucial in capturing learner knowledge states in
a consistent manner. Aside from crafting questions itself, putting together
relevant problems to form a question sheet is also a time-consuming process. In
this work, we present a generic formulation of question assembly and a genetic
algorithm based method that can generate assessment tests from raw
problem-solving history. First, we estimate the learner-question knowledge
matrix (snapshot). Each matrix element stands for the probability that a
learner correctly answers a specific question. We formulate the task as a
combinatorial search over this snapshot. To ensure representative and
discriminative diagnostic tests, questions are selected (1) that has a low root
mean squared error against the whole question pool and (2) high standard
deviation among learner performances. Experimental results show that the
proposed method outperforms greedy and random baseline by a large margin in one
private dataset and four public datasets. We also performed qualitative
analysis on the generated assessment test for 9th graders, which enjoys good
problem scatterness across the whole 9th grader curriculum and decent
difficulty level distribution.
Related papers
- Optimal Decision Tree with Noisy Outcomes [10.183589332901839]
In pool-based active learning, the learner is given an unlabeled data set and aims to efficiently learn the unknown hypothesis by querying the labels of the data points.
This can be formulated as the classical Optimal Decision Tree (ODT) problem: Given a set of tests, a set of hypotheses, and an outcome for each pair of test and hypothesis, our objective is to find a low-cost testing procedure (i.e., decision tree) that identifies the true hypothesis.
In this work, we study a fundamental variant of the ODT problem in which some test outcomes are noisy, even in the more general
arXiv Detail & Related papers (2023-12-23T21:47:50Z) - Precise Error Rates for Computationally Efficient Testing [75.63895690909241]
We revisit the question of simple-versus-simple hypothesis testing with an eye towards computational complexity.
An existing test based on linear spectral statistics achieves the best possible tradeoff curve between type I and type II error rates.
arXiv Detail & Related papers (2023-11-01T04:41:16Z) - TQ-Net: Mixed Contrastive Representation Learning For Heterogeneous Test
Questions [18.186909839033017]
Test questions (TQ) are usually heterogeneous and multi-modal, e.g., some of them may only contain text, while others half contain images with information beyond their literal description.
In this paper, we first improve previous text-only representation with a two-stage unsupervised instance level contrastive based pre-training method.
Then, TQ-Net was proposed to fuse the content of images to the representation of heterogeneous data.
arXiv Detail & Related papers (2023-03-09T10:55:48Z) - Multi-Label Quantification [78.83284164605473]
Quantification, variously called "labelled prevalence estimation" or "learning to quantify", is the supervised learning task of generating predictors of the relative frequencies of the classes of interest in unsupervised data samples.
We propose methods for inferring estimators of class prevalence values that strive to leverage the dependencies among the classes of interest in order to predict their relative frequencies more accurately.
arXiv Detail & Related papers (2022-11-15T11:29:59Z) - Statistical and Computational Phase Transitions in Group Testing [73.55361918807883]
We study the group testing problem where the goal is to identify a set of k infected individuals carrying a rare disease.
We consider two different simple random procedures for assigning individuals tests.
arXiv Detail & Related papers (2022-06-15T16:38:50Z) - Cognitive Diagnosis with Explicit Student Vector Estimation and
Unsupervised Question Matrix Learning [53.79108239032941]
We propose an explicit student vector estimation (ESVE) method to estimate the student vectors of DINA.
We also propose an unsupervised method called bidirectional calibration algorithm (HBCA) to label the Q-matrix automatically.
The experimental results on two real-world datasets show that ESVE-DINA outperforms the DINA model on accuracy and that the Q-matrix labeled automatically by HBCA can achieve performance comparable to that obtained with the manually labeled Q-matrix.
arXiv Detail & Related papers (2022-03-01T03:53:19Z) - Group Testing with Non-identical Infection Probabilities [59.96266198512243]
We develop an adaptive group testing algorithm using the set formation method.
We show that our algorithm outperforms the state of the art, and performs close to the entropy lower bound.
arXiv Detail & Related papers (2021-08-27T17:53:25Z) - Option Tracing: Beyond Correctness Analysis in Knowledge Tracing [3.1798318618973362]
We extend existing knowledge tracing methods to predict the exact option students select in multiple choice questions.
We quantitatively evaluate the performance of our option tracing methods on two large-scale student response datasets.
arXiv Detail & Related papers (2021-04-19T04:28:34Z) - Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
We consider fairness in the integration component of data management.
We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
arXiv Detail & Related papers (2020-06-10T20:20:10Z) - Discriminative Learning via Adaptive Questioning [6.378513792050356]
We consider the problem of designing an adaptive sequence of questions that optimally classify a candidate's ability into one of several categories or discriminative grades.
A candidate's ability is modeled as an unknown parameter, which, together with the difficulty of the question asked, determines the likelihood with which s/he is able to answer a question correctly.
arXiv Detail & Related papers (2020-04-11T16:50:00Z) - R2DE: a NLP approach to estimating IRT parameters of newly generated
questions [3.364554138758565]
R2DE is a model capable of assessing newly generated multiple-choice questions by looking at the text of the question.
In particular, it can estimate the difficulty and the discrimination of each question.
arXiv Detail & Related papers (2020-01-21T14:31:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.