Exploring a New Competency Modeling Process with Large Language Models

• URL: http://arxiv.org/abs/2602.13084v1
• Date: Fri, 13 Feb 2026 16:46:51 GMT
• Title: Exploring a New Competency Modeling Process with Large Language Models
• Authors: Silin Du, Manqing Xin, Raymond Jia Wang,
• Abstract summary: This study proposes a new competency modeling process built on large language models (LLMs)<n> Specifically, we leverage LLMs to extract behavioral and psychological descriptions from raw textual data.<n>We introduce a learnable parameter that adaptively integrates different information sources, enabling the model to determine the relative importance of behavioral and psychological signals.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Competency modeling is widely used in human resource management to select, develop, and evaluate talent. However, traditional expert-driven approaches rely heavily on manual analysis of large volumes of interview transcripts, making them costly and prone to randomness, ambiguity, and limited reproducibility. This study proposes a new competency modeling process built on large language models (LLMs). Instead of merely automating isolated steps, we reconstruct the workflow by decomposing expert practices into structured computational components. Specifically, we leverage LLMs to extract behavioral and psychological descriptions from raw textual data and map them to predefined competency libraries through embedding-based similarity. We further introduce a learnable parameter that adaptively integrates different information sources, enabling the model to determine the relative importance of behavioral and psychological signals. To address the long-standing challenge of validation, we develop an offline evaluation procedure that allows systematic model selection without requiring additional large-scale data collection. Empirical results from a real-world implementation in a software outsourcing company demonstrate strong predictive validity, cross-library consistency, and structural robustness. Overall, our framework transforms competency modeling from a largely qualitative and expert-dependent practice into a transparent, data-driven, and evaluable analytical process.

Related papers

• Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval [60.25608870901428]
  Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs)<n>We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source robustness.
  arXiv  Detail & Related papers  (2026-03-05T18:42:51Z)
• Large Language Model Sourcing: A Survey [84.63438376832471]
  Large language models (LLMs) have revolutionized artificial intelligence, shifting from supporting objective tasks to empowering subjective decision-making.<n>Due to the black-box nature of LLMs and the human-like quality of their generated content, issues such as hallucinations, bias, unfairness, and copyright infringement become significant.<n>This survey presents a systematic investigation into provenance tracking for content generated by LLMs, organized around four interrelated dimensions.
  arXiv  Detail & Related papers  (2025-10-11T10:52:30Z)
• Knowledge-Driven Hallucination in Large Language Models: An Empirical Study on Process Modeling [46.05103857535919]
  The utility of Large Language Models in analytical tasks is rooted in their vast pre-trained knowledge.<n>This same capability introduces a critical risk of what we term knowledge-driven hallucination.<n>This paper investigates this phenomenon by evaluating LLMs on the task of automated process modeling.
  arXiv  Detail & Related papers  (2025-09-18T18:27:30Z)
• A Survey of Model Architectures in Information Retrieval [59.61734783818073]
  The period from 2019 to the present has represented one of the biggest paradigm shifts in information retrieval (IR) and natural language processing (NLP)<n>We trace the development from traditional term-based methods to modern neural approaches, particularly highlighting the impact of transformer-based models and subsequent large language models (LLMs)<n>We conclude with a forward-looking discussion of emerging challenges and future directions.
  arXiv  Detail & Related papers  (2025-02-20T18:42:58Z)
• Large Language Models for Extrapolative Modeling of Manufacturing Processes [5.705795836910535]
  The novelty lies in combining automatic extraction of process-relevant knowledge embedded in the literature with iterative model refinement based on a small amount of experimental data.<n>The results show that for the same small experimental data budget the models derived by our framework have unexpectedly high extrapolative performance.
  arXiv  Detail & Related papers  (2025-02-15T02:43:22Z)
• Generating Computational Cognitive Models using Large Language Models [13.201934636532577]
  We introduce a pipeline for Guided generation of Computational Cognitive Models (GeCCo)<n>GeCCo prompts an LLM to propose candidate models, fits proposals to held-out data, and iteratively refines them based on their predictive performance.<n>We benchmark this approach across four different cognitive domains.
  arXiv  Detail & Related papers  (2025-02-02T19:07:13Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• Process Modeling With Large Language Models [42.0652924091318]
  This paper explores the integration of Large Language Models (LLMs) into process modeling. We propose a framework that leverages LLMs for the automated generation and iterative refinement of process models. Preliminary results demonstrate the framework's ability to streamline process modeling tasks.
  arXiv  Detail & Related papers  (2024-03-12T11:27:47Z)
• Learning to Extract Structured Entities Using Language Models [52.281701191329]
  Recent advances in machine learning have significantly impacted the field of information extraction. We reformulate the task to be entity-centric, enabling the use of diverse metrics. We contribute to the field by introducing Structured Entity Extraction and proposing the Approximate Entity Set OverlaP metric.
  arXiv  Detail & Related papers  (2024-02-06T22:15:09Z)
• QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
  We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights. We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
  arXiv  Detail & Related papers  (2023-11-06T00:21:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.