BOBCAT: Bilevel Optimization-Based Computerized Adaptive Testing

• URL: http://arxiv.org/abs/2108.07386v1
• Date: Tue, 17 Aug 2021 00:40:23 GMT
• Title: BOBCAT: Bilevel Optimization-Based Computerized Adaptive Testing
• Authors: Aritra Ghosh, Andrew Lan
• Abstract summary: Computerized adaptive testing (CAT) refers to a form of tests that are personalized to every student/test taker. We propose BOBCAT, a Bilevel Optimization-Based framework for CAT to directly learn a data-driven question selection algorithm from training data.
• Score: 3.756550107432323
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Computerized adaptive testing (CAT) refers to a form of tests that are personalized to every student/test taker. CAT methods adaptively select the next most informative question/item for each student given their responses to previous questions, effectively reducing test length. Existing CAT methods use item response theory (IRT) models to relate student ability to their responses to questions and static question selection algorithms designed to reduce the ability estimation error as quickly as possible; therefore, these algorithms cannot improve by learning from large-scale student response data. In this paper, we propose BOBCAT, a Bilevel Optimization-Based framework for CAT to directly learn a data-driven question selection algorithm from training data. BOBCAT is agnostic to the underlying student response model and is computationally efficient during the adaptive testing process. Through extensive experiments on five real-world student response datasets, we show that BOBCAT outperforms existing CAT methods (sometimes significantly) at reducing test length.

Related papers

• Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning [9.106234291496884]
  We propose a new data pruning technique: Checkpoints Across Time (CAT) We benchmark CAT against several data pruning techniques including COMET-QE, LASER and LaBSE. When applied to English-German, English-French and English-Swahili translation tasks, CAT achieves comparable performance to using the full dataset.
  arXiv  Detail & Related papers  (2024-05-29T19:21:49Z)
• Switchable Decision: Dynamic Neural Generation Networks [98.61113699324429]
  We propose a switchable decision to accelerate inference by dynamically assigning resources for each data instance. Our method benefits from less cost during inference while keeping the same accuracy.
  arXiv  Detail & Related papers  (2024-05-07T17:44:54Z)
• Survey of Computerized Adaptive Testing: A Machine Learning Perspective [66.26687542572974]
  Computerized Adaptive Testing (CAT) provides an efficient and tailored method for assessing the proficiency of examinees. This paper aims to provide a machine learning-focused survey on CAT, presenting a fresh perspective on this adaptive testing method.
  arXiv  Detail & Related papers  (2024-03-31T15:09:47Z)
• A Novel ML-driven Test Case Selection Approach for Enhancing the Performance of Grammatical Evolution [0.07499722271664144]
  We propose a Machine Learning-driven Distance-based Selection (DBS) algorithm that reduces the fitness evaluation time by optimizing test cases. We test our algorithm by applying it to 24 benchmark problems from Symbolic Regression (SR) and digital circuit domains and then using Grammatical Evolution (GE) to train models using the reduced dataset. The quality of the solutions is tested and compared against the conventional training method to measure the coverage of training data selected using DBS, i.e., how well the subset matches the statistical properties of the entire dataset.
  arXiv  Detail & Related papers  (2023-12-21T22:21:02Z)
• Addressing Selection Bias in Computerized Adaptive Testing: A User-Wise Aggregate Influence Function Approach [14.175555669521987]
  We propose a user-wise aggregate influence function method to tackle the selection bias issue. Our intuition is to filter out users whose response data is heavily biased in an aggregate manner.
  arXiv  Detail & Related papers  (2023-08-23T04:57:21Z)
• Balancing Test Accuracy and Security in Computerized Adaptive Testing [18.121437613260618]
  Bilevel optimization-based CAT (BOBCAT) is a framework that learns a data-driven question selection algorithm. It suffers from high question exposure and test overlap rates, which potentially affects test security. We show that C-BOBCAT is effective through extensive experiments on two real-world adult testing datasets.
  arXiv  Detail & Related papers  (2023-05-18T18:32:51Z)
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts [143.14128737978342]
  Test-time adaptation, an emerging paradigm, has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions. Recent progress in this paradigm highlights the significant benefits of utilizing unlabeled data for training self-adapted models prior to inference.
  arXiv  Detail & Related papers  (2023-03-27T16:32:21Z)
• DELTA: degradation-free fully test-time adaptation [59.74287982885375]
  We find that two unfavorable defects are concealed in the prevalent adaptation methodologies like test-time batch normalization (BN) and self-learning. First, we reveal that the normalization statistics in test-time BN are completely affected by the currently received test samples, resulting in inaccurate estimates. Second, we show that during test-time adaptation, the parameter update is biased towards some dominant classes.
  arXiv  Detail & Related papers  (2023-01-30T15:54:00Z)
• Quality meets Diversity: A Model-Agnostic Framework for Computerized Adaptive Testing [60.38182654847399]
  Computerized Adaptive Testing (CAT) is emerging as a promising testing application in many scenarios. We propose a novel framework, namely Model-Agnostic Adaptive Testing (MAAT) for CAT solution.
  arXiv  Detail & Related papers  (2021-01-15T06:48:50Z)
• CAT: Customized Adversarial Training for Improved Robustness [142.3480998034692]
  We propose a new algorithm, named Customized Adversarial Training (CAT), which adaptively customizes the perturbation level and the corresponding label for each training sample in adversarial training. We show that the proposed algorithm achieves better clean and robust accuracy than previous adversarial training methods through extensive experiments.
  arXiv  Detail & Related papers  (2020-02-17T06:13:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.