An Efficient Long-Context Ranking Architecture With Calibrated LLM Distillation: Application to Person-Job Fit

• URL: http://arxiv.org/abs/2601.10321v2
• Date: Fri, 16 Jan 2026 15:24:20 GMT
• Title: An Efficient Long-Context Ranking Architecture With Calibrated LLM Distillation: Application to Person-Job Fit
• Authors: Warren Jouanneau, Emma Jouffroy, Marc Palyart,
• Abstract summary: We propose a re-ranking model based on a new generation of late cross-attention architecture.<n>To mitigate historical data biases, we use a generative large language model (LLM) as a teacher.<n>The resulting model produces skill-fit scores that enable consistent and interpretable person-job matching.
• Score: 0.6117371161379207
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Finding the most relevant person for a job proposal in real time is challenging, especially when resumes are long, structured, and multilingual. In this paper, we propose a re-ranking model based on a new generation of late cross-attention architecture, that decomposes both resumes and project briefs to efficiently handle long-context inputs with minimal computational overhead. To mitigate historical data biases, we use a generative large language model (LLM) as a teacher, generating fine-grained, semantically grounded supervision. This signal is distilled into our student model via an enriched distillation loss function. The resulting model produces skill-fit scores that enable consistent and interpretable person-job matching. Experiments on relevance, ranking, and calibration metrics demonstrate that our approach outperforms state-of-the-art baselines.

Related papers

• Document Reconstruction Unlocks Scalable Long-Context RLVR [60.74632963522131]
  Reinforcement Learning with Verifiable Rewards(RLVR) has become a prominent paradigm to enhance the capabilities (i.e. long-context) of Large Language Models(LLMs)<n>We investigate unsupervised approaches to enhance the long-context capabilities of LLMs, eliminating the need for heavy human annotations or teacher models' supervision.<n>We validate the effectiveness of our method on two widely used benchmarks, RULER and LongBenchv2.
  arXiv  Detail & Related papers  (2026-02-09T03:23:23Z)
• Reading Between the Lines: Classifying Resume Seniority with Large Language Models [38.57404400070555]
  This study investigates the effectiveness of large language models for automating seniority classification in resumes.<n>We introduce a hybrid dataset comprising both real-world resumes and synthetically generated hard examples.<n>Using the dataset, we evaluate the performance of Large Language Models in detecting subtle linguistic cues associated with seniority inflation and implicit expertise.
  arXiv  Detail & Related papers  (2025-09-11T08:06:02Z)
• Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning [103.65680870130839]
  We investigate how to design instruction data for the post-training phase of a long context pre-trained model.<n>Our controlled study reveals that models instruction-tuned on short contexts can effectively generalize to longer ones.<n>Based on these findings, we propose context synthesis, a novel data synthesis framework.
  arXiv  Detail & Related papers  (2025-02-21T17:02:40Z)
• ACER: Automatic Language Model Context Extension via Retrieval [36.40066695682234]
  Current open-weight generalist long-context models are still lacking in practical long-context processing tasks. We build an textbfautomatic data synthesis pipeline that mimics this process using short-context LMs. The short-context LMs are further tuned using these self-generated data to obtain task-specific long-context capabilities.
  arXiv  Detail & Related papers  (2024-10-11T17:57:06Z)
• How to Train Long-Context Language Models (Effectively) [75.5418485597276]
  We study continued training and supervised fine-tuning (SFT) of a language model (LM) to make effective use of long-context information.<n>We find that code repositories and books are excellent sources of long data, but it is crucial to combine them with high-quality short-context data.<n>Our final model, ProLong-8B, demonstrates state-of-the-art long-context performance among similarly sized models at a length of 128K.
  arXiv  Detail & Related papers  (2024-10-03T16:46:52Z)
• Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models [13.091271774417867]
  Long-context modeling capabilities are important for large language models (LLMs) in various applications. We propose a data mining framework textbfProLong that can assign each training sample with a long dependency score. Comprehensive experiments on multiple benchmarks indicate that ProLong effectively identifies documents that carry long dependencies.
  arXiv  Detail & Related papers  (2024-05-28T07:36:56Z)
• List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation [80.12531449946655]
  We propose a Reranking-Truncation joint model (GenRT) that can perform the two tasks concurrently. GenRT integrates reranking and truncation via generative paradigm based on encoder-decoder architecture. Our method achieves SOTA performance on both reranking and truncation tasks for web search and retrieval-augmented LLMs.
  arXiv  Detail & Related papers  (2024-02-05T06:52:53Z)
• Structured Packing in LLM Training Improves Long Context Utilization [11.484631908171465]
  This study investigates structuring training data to enhance semantic interdependence.<n>We introduce the Structured Packing for Long Context (SPLiCe) method.<n>We validate SPLiCe empirically across models of varying sizes.
  arXiv  Detail & Related papers  (2023-12-28T16:25:52Z)
• Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks [8.235367170516769]
  We introduce the Resume-Job Description Benchmark (RJDB) to cater to a wide array of HR tasks. Our benchmark includes over 50 thousand triples of job descriptions, matched resumes and unmatched resumes. Our experiments reveal that the student models achieve near/better performance than the teacher model (GPT-4), affirming the effectiveness of the benchmark.
  arXiv  Detail & Related papers  (2023-11-10T20:25:42Z)
• Effective Long-Context Scaling of Foundation Models [90.57254298730923]
  We present a series of long-context LLMs that support effective context windows of up to 32,768 tokens. Our models achieve consistent improvements on most regular tasks and significant improvements on long-context tasks over Llama 2.
  arXiv  Detail & Related papers  (2023-09-27T21:41:49Z)
• A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis [90.24921443175514]
  We focus on aspect-based sentiment analysis, which involves extracting aspect term, category, and predicting their corresponding polarities. We propose to reformulate the extraction and prediction tasks into the sequence generation task, using a generative language model with unidirectional attention. Our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
  arXiv  Detail & Related papers  (2022-04-11T18:31:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.