Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks

• URL: http://arxiv.org/abs/2311.06383v1
• Date: Fri, 10 Nov 2023 20:25:42 GMT
• Title: Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks
• Authors: Pouya Pezeshkpour, Hayate Iso, Thom Lake, Nikita Bhutani, Estevam Hruschka
• Abstract summary: We introduce the Resume-Job Description Benchmark (RJDB) to cater to a wide array of HR tasks. Our benchmark includes over 50 thousand triples of job descriptions, matched resumes and unmatched resumes. Our experiments reveal that the student models achieve near/better performance than the teacher model (GPT-4), affirming the effectiveness of the benchmark.
• Score: 8.235367170516769
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Numerous HR applications are centered around resumes and job descriptions. While they can benefit from advancements in NLP, particularly large language models, their real-world adoption faces challenges due to absence of comprehensive benchmarks for various HR tasks, and lack of smaller models with competitive capabilities. In this paper, we aim to bridge this gap by introducing the Resume-Job Description Benchmark (RJDB). We meticulously craft this benchmark to cater to a wide array of HR tasks, including matching and explaining resumes to job descriptions, extracting skills and experiences from resumes, and editing resumes. To create this benchmark, we propose to distill domain-specific knowledge from a large language model (LLM). We rely on a curated skill-occupation graph to ensure diversity and provide context for LLMs generation. Our benchmark includes over 50 thousand triples of job descriptions, matched resumes and unmatched resumes. Using RJDB, we train multiple smaller student models. Our experiments reveal that the student models achieve near/better performance than the teacher model (GPT-4), affirming the effectiveness of the benchmark. Additionally, we explore the utility of RJDB on out-of-distribution data for skill extraction and resume-job description matching, in zero-shot and weak supervision manner. We release our datasets and code to foster further research and industry applications.

Related papers

• Systematic Task Exploration with LLMs: A Study in Citation Text Generation [63.50597360948099]
  Large language models (LLMs) bring unprecedented flexibility in defining and executing complex, creative natural language generation (NLG) tasks. We propose a three-component research framework that consists of systematic input manipulation, reference data, and output measurement. We use this framework to explore citation text generation -- a popular scholarly NLP task that lacks consensus on the task definition and evaluation metric.
  arXiv  Detail & Related papers  (2024-07-04T16:41:08Z)
• ResumeFlow: An LLM-facilitated Pipeline for Personalized Resume Generation and Refinement [14.044324268372847]
  We propose ResumeFlow: a Large Language Model (LLM) aided tool that enables an end user to simply provide their detailed resume and the desired job posting. Our proposed pipeline leverages the language understanding and information extraction capabilities of state-of-the-art LLMs such as OpenAI's GPT-4 and Google's Gemini. Our easy-to-use tool leverages the user-chosen LLM in a completely off-the-shelf manner, thus requiring no fine-tuning.
  arXiv  Detail & Related papers  (2024-02-09T07:13:44Z)
• CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation [94.59630161324013]
  We propose CoAnnotating, a novel paradigm for Human-LLM co-annotation of unstructured texts at scale. Our empirical study shows CoAnnotating to be an effective means to allocate work from results on different datasets, with up to 21% performance improvement over random baseline.
  arXiv  Detail & Related papers  (2023-10-24T08:56:49Z)
• Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistance [60.40541387785977]
  Small foundational models can display remarkable proficiency in tackling diverse tasks when fine-tuned using instruction-driven data. In this work, we investigate a practical problem setting where the primary focus is on one or a few particular tasks rather than general-purpose instruction following. Experimental results show that fine-tuning LLaMA on writing instruction data significantly improves its ability on writing tasks.
  arXiv  Detail & Related papers  (2023-05-22T16:56:44Z)
• AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators [98.11286353828525]
  GPT-3.5 series models have demonstrated remarkable few-shot and zero-shot ability across various NLP tasks. We propose AnnoLLM, which adopts a two-step approach, explain-then-annotate. We build the first conversation-based information retrieval dataset employing AnnoLLM.
  arXiv  Detail & Related papers  (2023-03-29T17:03:21Z)
• Design of Negative Sampling Strategies for Distantly Supervised Skill Extraction [19.43668931500507]
  We propose an end-to-end system for skill extraction, based on distant supervision through literal matching. We observe that using the ESCO taxonomy to select negative examples from related skills yields the biggest improvements. We release the benchmark dataset for research purposes to stimulate further research on the task.
  arXiv  Detail & Related papers  (2022-09-13T13:37:06Z)
• Construction of English Resume Corpus and Test with Pre-trained Language Models [0.0]
  This study aims to transform the information extraction task of resumes into a simple sentence classification task. The classification rules are improved to create a larger and more fine-grained classification dataset of resumes. This corpus is also used to test some current mainstream Pre-training language models (PLMs) performance.
  arXiv  Detail & Related papers  (2022-08-05T15:07:23Z)
• JobBERT: Understanding Job Titles through Skills [12.569546741576515]
  Job titles form a cornerstone of today's human resources (HR) processes. Job titles are a compact, convenient, and readily available data source. We propose a neural representation model for job titles, by augmenting a pre-trained language model with co-occurrence information from skill labels extracted from vacancies.
  arXiv  Detail & Related papers  (2021-09-20T15:00:10Z)
• conSultantBERT: Fine-tuned Siamese Sentence-BERT for Matching Jobs and Job Seekers [2.208694022993555]
  We explain our task where noisy data from parsed resumes, heterogeneous nature of the different sources of data, and crosslinguality and multilinguality present domain-specific challenges. We address these challenges by fine-tuning a Siamese Sentence Siamese-BERT (SBERT) model, which we call conSultantBERT, using a large-scale, real-world, and high quality dataset of over 270,000 resume-vacancy pairs labeled by our staffing consultants. We show how our fine-tuned model significantly outperforms unsupervised and supervised baselines that rely on TF-IDF-weighted feature vectors and BERT embeddings
  arXiv  Detail & Related papers  (2021-09-14T07:57:05Z)
• KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation [100.79870384880333]
  We propose a knowledge-grounded pre-training (KGPT) to generate knowledge-enriched text. We adopt three settings, namely fully-supervised, zero-shot, few-shot to evaluate its effectiveness. Under zero-shot setting, our model achieves over 30 ROUGE-L on WebNLG while all other baselines fail.
  arXiv  Detail & Related papers  (2020-10-05T19:59:05Z)
• Job2Vec: Job Title Benchmarking with Collective Multi-View Representation Learning [51.34011135329063]
  Job Title Benchmarking (JTB) aims at matching job titles with similar expertise levels across various companies. Traditional JTB approaches mainly rely on manual market surveys, which is expensive and labor-intensive. We reformulate the JTB as the task of link prediction over the Job-Graph that matched job titles should have links.
  arXiv  Detail & Related papers  (2020-09-16T02:33:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.