Trading off performance and human oversight in algorithmic policy: evidence from Danish college admissions

• URL: http://arxiv.org/abs/2411.15348v1
• Date: Fri, 22 Nov 2024 21:12:54 GMT
• Title: Trading off performance and human oversight in algorithmic policy: evidence from Danish college admissions
• Authors: Magnus Lindgaard Nielsen, Jonas Skjold Raaschou-Pedersen, Emil Chrisander, David Dreyer Lassen, Julien Grenet, Anna Rogers, Andreas Bjerre-Nielsen,
• Abstract summary: Student dropout is a significant concern for educational institutions. We show that sequential AI models offer more precise and fair predictions. We estimate that even the use of simple AI models to guide admissions decisions could yield significant economic benefits.
• Score: 11.378331161188022
• License:
• Abstract: Student dropout is a significant concern for educational institutions due to its social and economic impact, driving the need for risk prediction systems to identify at-risk students before enrollment. We explore the accuracy of such systems in the context of higher education by predicting degree completion before admission, with potential applications for prioritizing admissions decisions. Using a large-scale dataset from Danish higher education admissions, we demonstrate that advanced sequential AI models offer more precise and fair predictions compared to current practices that rely on either high school grade point averages or human judgment. These models not only improve accuracy but also outperform simpler models, even when the simpler models use protected sociodemographic attributes. Importantly, our predictions reveal how certain student profiles are better matched with specific programs and fields, suggesting potential efficiency and welfare gains in public policy. We estimate that even the use of simple AI models to guide admissions decisions, particularly in response to a newly implemented nationwide policy reducing admissions by 10 percent, could yield significant economic benefits. However, this improvement would come at the cost of reduced human oversight and lower transparency. Our findings underscore both the potential and challenges of incorporating advanced AI into educational policymaking.

Related papers

• Reduced-Rank Multi-objective Policy Learning and Optimization [57.978477569678844]
  In practice, causal researchers do not have a single outcome in mind a priori. In government-assisted social benefit programs, policymakers collect many outcomes to understand the multidimensional nature of poverty. We present a data-driven dimensionality-reduction methodology for multiple outcomes in the context of optimal policy learning.
  arXiv  Detail & Related papers  (2024-04-29T08:16:30Z)
• Temporal and Between-Group Variability in College Dropout Prediction [0.0]
  This study provides a systematic evaluation of contributing factors and predictive performance of machine learning models. We find dropout prediction at the end of the second year has a 20% higher AUC than at the time of enrollment in a Random Forest model. Regarding variability across student groups, college GPA has more predictive value for students from traditionally disadvantaged backgrounds than their peers.
  arXiv  Detail & Related papers  (2024-01-12T10:43:55Z)
• Fast Model Debias with Machine Unlearning [54.32026474971696]
  Deep neural networks might behave in a biased manner in many real-world scenarios. Existing debiasing methods suffer from high costs in bias labeling or model re-training. We propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases.
  arXiv  Detail & Related papers  (2023-10-19T08:10:57Z)
• Difficult Lessons on Social Prediction from Wisconsin Public Schools [32.90759447739759]
  Early warning systems assist in targeting interventions to individual students by predicting which students are at risk of dropping out. Despite significant investments in their widespread adoption, there remain large gaps in our understanding of the efficacy of EWS. We present empirical evidence that the prediction system accurately sorts students by their dropout risk. We find that it may have caused a single-digit percentage increase in graduation rates, though our empirical analyses cannot reliably rule out that there has been no positive treatment effect.
  arXiv  Detail & Related papers  (2023-04-13T00:59:12Z)
• ASPEST: Bridging the Gap Between Active Learning and Selective Prediction [56.001808843574395]
  Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples. In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
  arXiv  Detail & Related papers  (2023-04-07T23:51:07Z)
• Uncertainty-Aware Instance Reweighting for Off-Policy Learning [63.31923483172859]
  We propose a Uncertainty-aware Inverse Propensity Score estimator (UIPS) for improved off-policy learning. Experiment results on synthetic and three real-world recommendation datasets demonstrate the advantageous sample efficiency of the proposed UIPS estimator.
  arXiv  Detail & Related papers  (2023-03-11T11:42:26Z)
• Travel Demand Forecasting: A Fair AI Approach [0.9383397937755517]
  We propose a novel methodology to develop fairness-aware, highly-accurate travel demand forecasting models. Specifically, we introduce a new fairness regularization term, which is explicitly designed to measure the correlation between prediction accuracy and protected attributes. Results highlight that our proposed methodology can effectively enhance fairness for multiple protected attributes while preserving prediction accuracy.
  arXiv  Detail & Related papers  (2023-03-03T03:16:54Z)
• SF-PATE: Scalable, Fair, and Private Aggregation of Teacher Ensembles [50.90773979394264]
  This paper studies a model that protects the privacy of individuals' sensitive information while also allowing it to learn non-discriminatory predictors. A key characteristic of the proposed model is to enable the adoption of off-the-selves and non-private fair models to create a privacy-preserving and fair model.
  arXiv  Detail & Related papers  (2022-04-11T14:42:54Z)
• Should College Dropout Prediction Models Include Protected Attributes? [0.4125187280299248]
  We build machine learning models to predict student dropout after one academic year. We compare the overall performance and fairness of model predictions with or without four protected attributes. We find that including protected attributes does not impact the overall prediction performance and it only marginally improves algorithmic fairness of predictions.
  arXiv  Detail & Related papers  (2021-03-28T22:47:30Z)
• Learning Models for Actionable Recourse [31.30850378503406]
  We propose an algorithm that theoretically guarantees recourse to affected individuals with high probability without sacrificing accuracy. We demonstrate the efficacy of our approach via extensive experiments on real data.
  arXiv  Detail & Related papers  (2020-11-12T01:15:18Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.