Evaluating a Learned Admission-Prediction Model as a Replacement for
Standardized Tests in College Admissions
- URL: http://arxiv.org/abs/2302.03610v3
- Date: Tue, 23 May 2023 17:18:51 GMT
- Title: Evaluating a Learned Admission-Prediction Model as a Replacement for
Standardized Tests in College Admissions
- Authors: Hansol Lee, Ren\'e F. Kizilcec, Thorsten Joachims
- Abstract summary: College admissions offices have historically relied on standardized test scores to organize large applicant pools into viable subsets for review.
We explore a machine learning-based approach to replace the role of standardized tests in subset generation.
We find that a prediction model trained on past admission data outperforms an SAT-based model and matches the demographic composition of the last admitted class.
- Score: 21.70450099249114
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A growing number of college applications has presented an annual challenge
for college admissions in the United States. Admission offices have
historically relied on standardized test scores to organize large applicant
pools into viable subsets for review. However, this approach may be subject to
bias in test scores and selection bias in test-taking with recent trends toward
test-optional admission. We explore a machine learning-based approach to
replace the role of standardized tests in subset generation while taking into
account a wide range of factors extracted from student applications to support
a more holistic review. We evaluate the approach on data from an undergraduate
admission office at a selective US institution (13,248 applications). We find
that a prediction model trained on past admission data outperforms an SAT-based
heuristic and matches the demographic composition of the last admitted class.
We discuss the risks and opportunities for how such a learned model could be
leveraged to support human decision-making in college admissions.
Related papers
- Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability [18.289154814012996]
We show that removing race data from a developed applicant ranking algorithm reduces the diversity of the top-ranked pool without meaningfully increasing the academic merit of that pool.
We measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness.
arXiv Detail & Related papers (2024-06-24T14:59:30Z) - VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model [72.13121434085116]
VLBiasBench is a benchmark aimed at evaluating biases in Large Vision-Language Models (LVLMs)
We construct a dataset encompassing nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status and two intersectional bias categories (race x gender, and race x social economic status)
We conduct extensive evaluations on 15 open-source models as well as one advanced closed-source model, providing some new insights into the biases revealing from these models.
arXiv Detail & Related papers (2024-06-20T10:56:59Z) - Semantic Properties of cosine based bias scores for word embeddings [52.13994416317707]
We propose requirements for bias scores to be considered meaningful for quantifying biases.
We analyze cosine based scores from the literature with regard to these requirements.
We underline these findings with experiments to show that the bias scores' limitations have an impact in the application case.
arXiv Detail & Related papers (2024-01-27T20:31:10Z) - Admission Prediction in Undergraduate Applications: an Interpretable
Deep Learning Approach [0.6906005491572401]
This article addresses the challenge of validating the admission committee's decisions for undergraduate admissions.
We propose deep learning-based classifiers, namely Feed-Forward and Input Convex neural networks.
Our models achieve higher accuracy compared to the best-performing traditional machine learning-based approach by a considerable margin of 3.03%.
arXiv Detail & Related papers (2024-01-22T05:44:43Z) - AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models [122.63704560157909]
We introduce AGIEval, a novel benchmark designed to assess foundation model in the context of human-centric standardized exams.
We evaluate several state-of-the-art foundation models, including GPT-4, ChatGPT, and Text-Davinci-003.
GPT-4 surpasses average human performance on SAT, LSAT, and math competitions, attaining a 95% accuracy rate on the SAT Math test and a 92.5% accuracy on the English test of the Chinese national college entrance exam.
arXiv Detail & Related papers (2023-04-13T09:39:30Z) - A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts [143.14128737978342]
Test-time adaptation, an emerging paradigm, has the potential to adapt a pre-trained model to unlabeled data during testing, before making predictions.
Recent progress in this paradigm highlights the significant benefits of utilizing unlabeled data for training self-adapted models prior to inference.
arXiv Detail & Related papers (2023-03-27T16:32:21Z) - Testing Occupational Gender Bias in Language Models: Towards Robust Measurement and Zero-Shot Debiasing [98.07536837448293]
Large language models (LLMs) have been shown to exhibit a variety of harmful, human-like biases against various demographics.
We introduce a list of desiderata for robustly measuring biases in generative language models.
We then use this benchmark to test several state-of-the-art open-source LLMs, including Llama, Mistral, and their instruction-tuned versions.
arXiv Detail & Related papers (2022-12-20T22:41:24Z) - The SAME score: Improved cosine based bias score for word embeddings [63.24247894974291]
We provide a bias definition based on the ideas from the literature and derive novel requirements for bias scores.
We propose a new bias score, SAME, to address the shortcomings of existing bias scores and show empirically that SAME is better suited to quantify biases in word embeddings.
arXiv Detail & Related papers (2022-03-28T09:28:13Z) - Using a Binary Classification Model to Predict the Likelihood of
Enrolment to the Undergraduate Program of a Philippine University [0.0]
This study covered an analysis of various characteristics of freshmen applicants affecting their admission status in a Philippine university.
A predictive model was developed using Logistic Regression to evaluate the probability that an admitted student will pursue to enroll in the Institution or not.
arXiv Detail & Related papers (2020-10-26T06:58:03Z) - Intersectional Affirmative Action Policies for Top-k Candidates
Selection [3.4961413413444817]
We study the problem of selecting the top-k candidates from a pool of applicants, where each candidate is associated with a score indicating his/her aptitude.
We consider a situation in which some groups of candidates experience historical and present disadvantage that makes their chances of being accepted much lower than other groups.
We propose two algorithms to solve this problem, analyze them, and evaluate them experimentally using a dataset of university application scores and admissions to bachelor degrees in an OECD country.
arXiv Detail & Related papers (2020-07-29T12:27:18Z) - Towards Data-Driven Affirmative Action Policies under Uncertainty [3.9293125023197595]
We consider affirmative action policies that seek to increase the number of admitted applicants from underrepresented groups.
Since such a policy has to be announced before the start of the application period, there is uncertainty about the score distribution of the students applying to each program.
We explore the possibility of using a predictive model trained on historical data to help optimize the parameters of such policies.
arXiv Detail & Related papers (2020-07-02T15:37:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.