Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability

• URL: http://arxiv.org/abs/2407.11199v1
• Date: Mon, 24 Jun 2024 14:59:30 GMT
• Title: Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability
• Authors: Jinsook Lee, Emma Harvey, Joyce Zhou, Nikhil Garg, Thorsten Joachims, Rene F. Kizilcec,
• Abstract summary: We show that removing race data from a developed applicant ranking algorithm reduces the diversity of the top-ranked pool without meaningfully increasing the academic merit of that pool. We measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness.
• Score: 18.289154814012996
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Each year, selective American colleges sort through tens of thousands of applications to identify a first-year class that displays both academic merit and diversity. In the 2023-2024 admissions cycle, these colleges faced unprecedented challenges. First, the number of applications has been steadily growing. Second, test-optional policies that have remained in place since the COVID-19 pandemic limit access to key information historically predictive of academic success. Most recently, longstanding debates over affirmative action culminated in the Supreme Court banning race-conscious admissions. Colleges have explored machine learning (ML) models to address the issues of scale and missing test scores, often via ranking algorithms intended to focus on 'top' applicants. However, the Court's ruling will force changes to these models, which were able to consider race as a factor in ranking. There is currently a poor understanding of how these mandated changes will shape applicant ranking algorithms, and, by extension, admitted classes. We seek to address this by quantifying the impact of different admission policies on the applications prioritized for review. We show that removing race data from a developed applicant ranking algorithm reduces the diversity of the top-ranked pool without meaningfully increasing the academic merit of that pool. We contextualize this impact by showing that excluding data on applicant race has a greater impact than excluding other potentially informative variables like intended majors. Finally, we measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness. We find that any given policy has a high degree of arbitrariness and that removing race data from the ranking algorithm increases arbitrariness in outcomes for most applicants.

Related papers

• On the Use of Proxies in Political Ad Targeting [49.61009579554272]
  We show that major political advertisers circumvented mitigations by targeting proxy attributes. Our findings have crucial implications for the ongoing discussion on the regulation of political advertising.
  arXiv  Detail & Related papers  (2024-10-18T17:15:13Z)
• Auditing for Racial Discrimination in the Delivery of Education Ads [50.37313459134418]
  We propose a new third-party auditing method that can evaluate racial bias in the delivery of ads for education opportunities. We find evidence of racial discrimination in Meta's algorithmic delivery of ads for education opportunities, posing legal and ethical concerns.
  arXiv  Detail & Related papers  (2024-06-02T02:00:55Z)
• Regularization-Based Methods for Ordinal Quantification [49.606912965922504]
  We study the ordinal case, i.e., the case in which a total order is defined on the set of n>2 classes. We propose a novel class of regularized OQ algorithms, which outperforms existing algorithms in our experiments.
  arXiv  Detail & Related papers  (2023-10-13T16:04:06Z)
• FRAMM: Fair Ranking with Missing Modalities for Clinical Trial Site Selection [55.2629939137135]
  This paper focuses on the trial site selection task and proposes FRAMM, a deep reinforcement learning framework for fair trial site selection. We focus on addressing two real-world challenges that affect fair trial sites selection: the data modalities are often not complete for many potential trial sites, and the site selection needs to simultaneously optimize for both enrollment and diversity. We evaluate FRAMM using 4,392 real-world clinical trials ranging from 2016 to 2021 and show that FRAMM outperforms the leading baseline in enrollment-only settings.
  arXiv  Detail & Related papers  (2023-05-30T20:44:14Z)
• Evaluating a Learned Admission-Prediction Model as a Replacement for Standardized Tests in College Admissions [21.70450099249114]
  College admissions offices have historically relied on standardized test scores to organize large applicant pools into viable subsets for review. We explore a machine learning-based approach to replace the role of standardized tests in subset generation. We find that a prediction model trained on past admission data outperforms an SAT-based model and matches the demographic composition of the last admitted class.
  arXiv  Detail & Related papers  (2023-02-07T17:14:26Z)
• Integrating Rankings into Quantized Scores in Peer Review [61.27794774537103]
  In peer review, reviewers are usually asked to provide scores for the papers. To mitigate this issue, conferences have started to ask reviewers to additionally provide a ranking of the papers they have reviewed. There are no standard procedure for using this ranking information and Area Chairs may use it in different ways. We take a principled approach to integrate the ranking information into the scores.
  arXiv  Detail & Related papers  (2022-04-05T19:39:13Z)
• An Outcome Test of Discrimination for Ranked Lists [0.18416014644193063]
  We show that non-discrimination implies a system of moment inequalities. We show how to statistically test the implied inequalities, and validate our approach in an application using data from LinkedIn.
  arXiv  Detail & Related papers  (2021-11-15T16:42:57Z)
• PiRank: Learning To Rank via Differentiable Sorting [85.28916333414145]
  We propose PiRank, a new class of differentiable surrogates for ranking. We show that PiRank exactly recovers the desired metrics in the limit of zero temperature.
  arXiv  Detail & Related papers  (2020-12-12T05:07:36Z)
• Intersectional Affirmative Action Policies for Top-k Candidates Selection [3.4961413413444817]
  We study the problem of selecting the top-k candidates from a pool of applicants, where each candidate is associated with a score indicating his/her aptitude. We consider a situation in which some groups of candidates experience historical and present disadvantage that makes their chances of being accepted much lower than other groups. We propose two algorithms to solve this problem, analyze them, and evaluate them experimentally using a dataset of university application scores and admissions to bachelor degrees in an OECD country.
  arXiv  Detail & Related papers  (2020-07-29T12:27:18Z)
• Towards Data-Driven Affirmative Action Policies under Uncertainty [3.9293125023197595]
  We consider affirmative action policies that seek to increase the number of admitted applicants from underrepresented groups. Since such a policy has to be announced before the start of the application period, there is uncertainty about the score distribution of the students applying to each program. We explore the possibility of using a predictive model trained on historical data to help optimize the parameters of such policies.
  arXiv  Detail & Related papers  (2020-07-02T15:37:16Z)
• Sorting Big Data by Revealed Preference with Application to College Ranking [0.0]
  When ranking big data observations, diverse consumers reveal heterogeneous preferences. A properly sorted solution could help consumers make the right choices, and governments make wise policy decisions. The employed approach can be applied in many other areas, such as sports team ranking, academic journal ranking, voting, and real effective exchange rates.
  arXiv  Detail & Related papers  (2020-03-27T01:11:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.