Predicting Skill Shortages in Labor Markets: A Machine Learning Approach
- URL: http://arxiv.org/abs/2004.01311v3
- Date: Wed, 26 Aug 2020 04:06:25 GMT
- Title: Predicting Skill Shortages in Labor Markets: A Machine Learning Approach
- Authors: Nik Dawson, Marian-Andrei Rizoiu, Benjamin Johnston and Mary-Anne
Williams
- Abstract summary: This research implements a high-performing Machine Learning approach to predict occupational skill shortages.
We compile a unique dataset of both Labor Demand and Labor Supply occupational data in Australia from 2012 to 2018.
Job ads data and employment statistics were the highest performing feature sets for predicting year-to-year skills shortage changes for occupations.
- Score: 7.503338065129185
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Skill shortages are a drain on society. They hamper economic opportunities
for individuals, slow growth for firms, and impede labor productivity in
aggregate. Therefore, the ability to understand and predict skill shortages in
advance is critical for policy-makers and educators to help alleviate their
adverse effects. This research implements a high-performing Machine Learning
approach to predict occupational skill shortages. In addition, we demonstrate
methods to analyze the underlying skill demands of occupations in shortage and
the most important features for predicting skill shortages. For this work, we
compile a unique dataset of both Labor Demand and Labor Supply occupational
data in Australia from 2012 to 2018. This includes data from 7.7 million job
advertisements (ads) and 20 official labor force measures. We use these data as
explanatory variables and leverage the XGBoost classifier to predict yearly
skills shortage classifications for 132 standardized occupations. The models we
construct achieve macro-F1 average performance scores of up to 83 per cent. Our
results show that job ads data and employment statistics were the highest
performing feature sets for predicting year-to-year skills shortage changes for
occupations. We also find that features such as 'Hours Worked', years of
'Education', years of 'Experience', and median 'Salary' are highly important
features for predicting occupational skill shortages. This research provides a
robust data-driven approach for predicting and analyzing skill shortages, which
can assist policy-makers, educators, and businesses to prepare for the future
of work.
Related papers
- Performative Prediction on Games and Mechanism Design [69.7933059664256]
We study a collective risk dilemma where agents decide whether to trust predictions based on past accuracy.
As predictions shape collective outcomes, social welfare arises naturally as a metric of concern.
We show how to achieve better trade-offs and use them for mechanism design.
arXiv Detail & Related papers (2024-08-09T16:03:44Z) - F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data [65.6499834212641]
We formulate the demand prediction as a meta-learning problem and develop the Feature-based First-Order Model-Agnostic Meta-Learning (F-FOMAML) algorithm.
By considering domain similarities through task-specific metadata, our model improved generalization, where the excess risk decreases as the number of training tasks increases.
Compared to existing state-of-the-art models, our method demonstrates a notable improvement in demand prediction accuracy, reducing the Mean Absolute Error by 26.24% on an internal vending machine dataset and by 1.04% on the publicly accessible JD.com dataset.
arXiv Detail & Related papers (2024-06-23T21:28:50Z) - Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking [59.87055275344965]
Job-SDF is a dataset designed to train and benchmark job-skill demand forecasting models.
Based on 10.35 million public job advertisements collected from major online recruitment platforms in China between 2021 and 2023.
Our dataset uniquely enables evaluating skill demand forecasting models at various granularities, including occupation, company, and regional levels.
arXiv Detail & Related papers (2024-06-17T07:22:51Z) - ASPEST: Bridging the Gap Between Active Learning and Selective
Prediction [56.001808843574395]
Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain.
Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples.
In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
arXiv Detail & Related papers (2023-04-07T23:51:07Z) - Long-term dynamics of fairness: understanding the impact of data-driven
targeted help on job seekers [1.357291726431012]
We use an approach that combines statistics and machine learning to assess long-term fairness effects of labor market interventions.
We develop and use a model to investigate the impact of decisions caused by a public employment authority that selectively supports job-seekers.
We conclude that in order to quantify the trade-off correctly and to assess the long-term fairness effects of such a system in the real-world, careful modeling of the surrounding labor market is indispensable.
arXiv Detail & Related papers (2022-08-17T12:03:23Z) - Forecasting Future World Events with Neural Networks [68.43460909545063]
Autocast is a dataset containing thousands of forecasting questions and an accompanying news corpus.
The news corpus is organized by date, allowing us to precisely simulate the conditions under which humans made past forecasts.
We test language models on our forecasting task and find that performance is far below a human expert baseline.
arXiv Detail & Related papers (2022-06-30T17:59:14Z) - What Should I Know? Using Meta-gradient Descent for Predictive Feature
Discovery in a Single Stream of Experience [63.75363908696257]
computational reinforcement learning seeks to construct an agent's perception of the world through predictions of future sensations.
An open challenge in this line of work is determining from the infinitely many predictions that the agent could possibly make which predictions might best support decision-making.
We introduce a meta-gradient descent process by which an agent learns what predictions to make, 2) the estimates for its chosen predictions, and 3) how to use those estimates to generate policies that maximize future reward.
arXiv Detail & Related papers (2022-06-13T21:31:06Z) - Practical Skills Demand Forecasting via Representation Learning of
Temporal Dynamics [4.536775100566484]
Rapid technological innovation threatens to leave much of the global workforce behind.
Governments and markets must find ways to quicken the rate at which the supply of skills reacts to changes in demand.
This paper presents a pipeline which makes one-shot multi-step forecasts into the future using a decade of monthly skill demand observations.
arXiv Detail & Related papers (2022-05-18T04:02:55Z) - Fairness in Algorithmic Profiling: A German Case Study [0.0]
We compare and evaluate statistical models for predicting job seekers' risk of becoming long-term unemployed.
We show that these models can be used to predict long-term unemployment with competitive levels of accuracy.
We highlight that different classification policies have very different fairness implications.
arXiv Detail & Related papers (2021-08-04T13:43:42Z) - Low-skilled Occupations Face the Highest Upskilling Pressure [0.0]
We examine how job contents evolve as new technologies substitute for tasks, shifting required skills rather than eliminating entire jobs.
We find that re-skilling pressure is greatest for low-skilled occupations when accounting for distance between skills.
Jobs from large employers and markets experienced less change relative to small employers and markets, and non-white workers in low-skilled jobs are most demographically vulnerable.
arXiv Detail & Related papers (2021-01-27T16:02:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.