JobPulse: A Big Data Approach to Real-Time Engineering Workforce Analysis and National Industrial Policy
- URL: http://arxiv.org/abs/2508.11014v1
- Date: Thu, 14 Aug 2025 18:36:55 GMT
- Title: JobPulse: A Big Data Approach to Real-Time Engineering Workforce Analysis and National Industrial Policy
- Authors: Karen S. Markel, Mihir Tale, Andrea Belz,
- Abstract summary: We use web scraping tools and a new data processing scheme to build a job posting data set for the semiconductor industry.<n>We report on the employer base and relative needs of various job functions.
- Score: 0.0
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Employment on a societal scale contributes heavily to national and global affairs; consequently, job openings and unemployment estimates provide important information to financial markets and governments alike. However, such reports often describe only the supply (employee job seeker) side of the job market, and skill mismatches are poorly understood. Job postings aggregated on recruiting platforms illuminate marketplace demand, but to date have primarily focused on candidate skills described in their personal profiles. In this paper, we report on a big data approach to estimating job market mismatches by focusing on demand, as represented in publicly available job postings. We use commercially available web scraping tools and a new data processing scheme to build a job posting data set for the semiconductor industry, a strategically critical sector of the United States economy; we focus on Southern California as a central hub of advanced technologies. We report on the employer base and relative needs of various job functions. Our work contributes on three fronts: First, we provide nearly real-time insight into workforce demand; second, we discuss disambiguation and semantic challenges in analysis of employer data bases at scale; and third, we report on the Southern California semiconductor engineering ecosystem.
Related papers
- Can Online GenAI Discussion Serve as Bellwether for Labor Market Shifts? [62.386835769570006]
This paper examines whether online discussions about Large Language Models can function as early indicators of labor market shifts.<n>We employ four distinct analytical approaches to identify the domains and timeframes in which public discourse serves as a leading signal for employment changes.<n>Our findings reveal that discussion intensity predicts employment changes 1-7 months in advance across multiple indicators, including job postings, net hiring rates, tenure patterns, and unemployment duration.
arXiv Detail & Related papers (2025-11-20T04:18:25Z) - JobHop: A Large-Scale Dataset of Career Trajectories [48.881023210777585]
JobHop is a large-scale public dataset derived from anonymized resumes provided by VDAB, the public employment service in Flanders, Belgium.<n>We process unstructured resume data to extract structured career information, which is then mapped to standardized ESCO occupation codes.<n>This results in a rich dataset of over 2.3 million work experiences, extracted from and grouped into more than 391,000 user resumes.
arXiv Detail & Related papers (2025-05-12T15:22:29Z) - Nasdaq-100 Companies' Hiring Insights: A Topic-based Classification Approach to the Labor Market [0.0]
We propose a data mining-based approach for job classification in the modern online labor market.
Among all 13 job categories, Marketing, Branding, and Sales; Software Engineering; Hardware Engineering; Industrial Engineering; and Project Management are the most frequently posted job classifications.
arXiv Detail & Related papers (2024-09-01T08:18:56Z) - Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking [59.87055275344965]
Job-SDF is a dataset designed to train and benchmark job-skill demand forecasting models.<n>Based on 10.35 million public job advertisements collected from major online recruitment platforms in China between 2021 and 2023.<n>Our dataset uniquely enables evaluating skill demand forecasting models at various granularities, including occupation, company, and regional levels.
arXiv Detail & Related papers (2024-06-17T07:22:51Z) - Professional Network Matters: Connections Empower Person-Job Fit [62.20651880558674]
This paper emphasizes the importance of incorporating professional networks into the Person-Job Fit model.
We introduce a job-specific attention mechanism in CSAGNN to handle noisy professional networks.
We demonstrate the effectiveness of our approach through experimental evaluations conducted across three real-world recruitment datasets from LinkedIn.
arXiv Detail & Related papers (2023-12-19T06:44:44Z) - A practical method for occupational skills detection in Vietnamese job
listings [0.16114012813668932]
Lack of accurate and timely labor market information leads to skill miss-matches.
Traditional approaches rely on existing taxonomy and/or large annotated data.
We propose a practical methodology for skill detection in Vietnamese job listings.
arXiv Detail & Related papers (2022-10-26T10:23:18Z) - Understanding Information Disclosure from Secure Computation Output: A Study of Average Salary Computation [58.74407460023331]
Quantifying information disclosure about private inputs from observing a function outcome is the subject of this work.
Motivated by the City of Boston gender pay gap studies, in this work we focus on the computation of the average of salaries.
arXiv Detail & Related papers (2022-09-21T15:59:48Z) - Toward Knowledge Discovery Framework for Data Science Job Market in the
United States [1.7205106391379024]
This paper introduces a framework to analyze the job market for data science-related jobs within the US.
The proposed framework includes three sub-modules allowing continuous data collection, information extraction, and a web-based visualization dashboard.
The current version of this application is deployed on the web and allows individuals and institutes to investigate skills required for data science positions.
arXiv Detail & Related papers (2021-06-14T21:23:15Z) - DataOps for Societal Intelligence: a Data Pipeline for Labor Market
Skills Extraction and Matching [5.842787579447653]
We formulate and solve this problem using DataOps models.
We then focus on the critical task of skills extraction from resumes.
We showcase preliminary results with applied machine learning on real data.
arXiv Detail & Related papers (2021-04-05T15:37:25Z) - Job2Vec: Job Title Benchmarking with Collective Multi-View
Representation Learning [51.34011135329063]
Job Title Benchmarking (JTB) aims at matching job titles with similar expertise levels across various companies.
Traditional JTB approaches mainly rely on manual market surveys, which is expensive and labor-intensive.
We reformulate the JTB as the task of link prediction over the Job-Graph that matched job titles should have links.
arXiv Detail & Related papers (2020-09-16T02:33:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.