Evaluating a Learned Admission-Prediction Model as a Replacement for Standardized Tests in College Admissions

• URL: http://arxiv.org/abs/2302.03610v3
• Date: Tue, 23 May 2023 17:18:51 GMT
• Title: Evaluating a Learned Admission-Prediction Model as a Replacement for Standardized Tests in College Admissions
• Authors: Hansol Lee, Ren\'e F. Kizilcec, Thorsten Joachims
• Abstract summary: College admissions offices have historically relied on standardized test scores to organize large applicant pools into viable subsets for review. We explore a machine learning-based approach to replace the role of standardized tests in subset generation. We find that a prediction model trained on past admission data outperforms an SAT-based model and matches the demographic composition of the last admitted class.
• Score: 21.70450099249114
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A growing number of college applications has presented an annual challenge for college admissions in the United States. Admission offices have historically relied on standardized test scores to organize large applicant pools into viable subsets for review. However, this approach may be subject to bias in test scores and selection bias in test-taking with recent trends toward test-optional admission. We explore a machine learning-based approach to replace the role of standardized tests in subset generation while taking into account a wide range of factors extracted from student applications to support a more holistic review. We evaluate the approach on data from an undergraduate admission office at a selective US institution (13,248 applications). We find that a prediction model trained on past admission data outperforms an SAT-based heuristic and matches the demographic composition of the last admitted class. We discuss the risks and opportunities for how such a learned model could be leveraged to support human decision-making in college admissions.

Related papers

• Bias Analysis of AI Models for Undergraduate Student Admissions [0.0]
  This work extends previous research done by the authors to provide a rigorous and more complete analysis of the bias found in AI predictive models. Data spanning six years was used to create an AI model to determine whether a given student would be directly admitted into the School of Science. We developed and analyzed AI models to understand which variables are important in admissions decisions, and how the decision to exclude test scores affects the demographics of the students who are admitted.
  arXiv  Detail & Related papers  (2024-12-03T16:21:37Z)
• Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models [49.06068319380296]
  We introduce context-aware testing (CAT) which uses context as an inductive bias to guide the search for meaningful model failures. We instantiate the first CAT system, SMART Testing, which employs large language models to hypothesize relevant and likely failures.
  arXiv  Detail & Related papers  (2024-10-31T15:06:16Z)
• Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability [18.289154814012996]
  We show that removing race data from a developed applicant ranking algorithm reduces the diversity of the top-ranked pool without meaningfully increasing the academic merit of that pool. We measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness.
  arXiv  Detail & Related papers  (2024-06-24T14:59:30Z)
• VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model [72.13121434085116]
  We introduce VLBiasBench, a benchmark to evaluate biases in Large Vision-Language Models (LVLMs) VLBiasBench features a dataset that covers nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status, as well as two intersectional bias categories: race x gender and race x social economic status. We conduct extensive evaluations on 15 open-source models as well as two advanced closed-source models, yielding new insights into the biases present in these models.
  arXiv  Detail & Related papers  (2024-06-20T10:56:59Z)
• Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing Platform [64.76104135495576]
  We propose a novel anonymous crowd-sourcing evaluation platform, BingJian, for large language models. Through this platform, users have the opportunity to submit their questions, testing the models on a personalized and potentially broader range of capabilities.
  arXiv  Detail & Related papers  (2024-03-13T07:31:20Z)
• Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation [49.3814117521631]
  Standard benchmarks of bias and fairness in large language models (LLMs) measure the association between social attributes implied in user prompts and short responses. We develop analogous RUTEd evaluations from three contexts of real-world use. We find that standard bias metrics have no significant correlation with the more realistic bias metrics.
  arXiv  Detail & Related papers  (2024-02-20T01:49:15Z)
• Admission Prediction in Undergraduate Applications: an Interpretable Deep Learning Approach [0.6906005491572401]
  This article addresses the challenge of validating the admission committee's decisions for undergraduate admissions. We propose deep learning-based classifiers, namely Feed-Forward and Input Convex neural networks. Our models achieve higher accuracy compared to the best-performing traditional machine learning-based approach by a considerable margin of 3.03%.
  arXiv  Detail & Related papers  (2024-01-22T05:44:43Z)
• AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models [122.63704560157909]
  We introduce AGIEval, a novel benchmark designed to assess foundation model in the context of human-centric standardized exams. We evaluate several state-of-the-art foundation models, including GPT-4, ChatGPT, and Text-Davinci-003. GPT-4 surpasses average human performance on SAT, LSAT, and math competitions, attaining a 95% accuracy rate on the SAT Math test and a 92.5% accuracy on the English test of the Chinese national college entrance exam.
  arXiv  Detail & Related papers  (2023-04-13T09:39:30Z)
• A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts [117.72709110877939]
  Test-time adaptation (TTA) has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions. We categorize TTA into several distinct groups based on the form of test data, namely, test-time domain adaptation, test-time batch adaptation, and online test-time adaptation.
  arXiv  Detail & Related papers  (2023-03-27T16:32:21Z)
• Using a Binary Classification Model to Predict the Likelihood of Enrolment to the Undergraduate Program of a Philippine University [0.0]
  This study covered an analysis of various characteristics of freshmen applicants affecting their admission status in a Philippine university. A predictive model was developed using Logistic Regression to evaluate the probability that an admitted student will pursue to enroll in the Institution or not.
  arXiv  Detail & Related papers  (2020-10-26T06:58:03Z)
• Towards Data-Driven Affirmative Action Policies under Uncertainty [3.9293125023197595]
  We consider affirmative action policies that seek to increase the number of admitted applicants from underrepresented groups. Since such a policy has to be announced before the start of the application period, there is uncertainty about the score distribution of the students applying to each program. We explore the possibility of using a predictive model trained on historical data to help optimize the parameters of such policies.
  arXiv  Detail & Related papers  (2020-07-02T15:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.