Comparing Retrieval Methods for Academic Advisor Discovery: A Six-Method Study of 768 CS Faculty Profiles Across 9 US Universities

• URL: http://arxiv.org/abs/2609.03901v1
• Date: Thu, 03 Sep 2026 14:19:14 GMT
• Title: Comparing Retrieval Methods for Academic Advisor Discovery: A Six-Method Study of 768 CS Faculty Profiles Across 9 US Universities
• Abstract summary: Methods span sparse lexical matching, TF-IDF, BM25, dense semantic retrieval, hybrid score fusion, and learning-to-rank.<n>We present a comparative evaluation of six information retrieval methods for the task of academic advisor discovery.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a comparative evaluation of six information retrieval methods for the task of academic advisor discovery: ranking CS faculty members by relevance to a graduate applicant's research interest statement. The methods span sparse lexical matching (Jaccard overlap, TF-IDF, BM25), dense semantic retrieval (all-MiniLM-L6-v2 sentence embeddings), hybrid score fusion, and learning-to-rank. Evaluation uses a new domain-specific collection: 768 faculty profiles scraped from 9 US CS departments, with 162 graded relevance judgments (grade 0/1/2) across 5 queries representing distinct graduate student research profiles. Across all five queries, Reranked achieves the highest mean NDCG@10 (0.477, std 0.138), followed by Semantic (0.450), Hybrid (0.421), BM25 (0.406), Jaccard (0.303), and TF-IDF (0.246). After Bonferroni correction across all 15 pairwise comparisons, TF-IDF is significantly worse than BM25, Semantic, Hybrid, and Reranked; no other pairwise difference survives correction at 5 queries. A field ablation reveals that biography alone (NDCG 0.634) outperforms the full model combining biography with research area tags (0.593). A controlled experiment shows that concatenating arXiv paper abstracts reduces NDCG@10 by 0.176, motivating a late-fusion architecture. All code, scrapers, and relevance labels are released openly.

Related papers

• BOUND: Brief-Guided Corrective Preference Distillation at Search-Control Boundaries [82.41764922522565]
  BOUND is a brief-guided corrective preference distillation framework for persistent search drift.<n>It outperforms Trajectory SFT by 5.6 EM points on Bamboogle and 4.8 accuracy points on BrowseComp-Plus.
  arXiv  Detail & Related papers  (2026-08-09T15:32:16Z)
• Trust Before Fusion: QIMG-7 and Source-Aware Resolution for Polluted Multimodal RAG [45.88028371034407]
  Multimodal retrieval-augmented generation (RAG) is often evaluated with clean evidence, yet real retrieval can return topically relevant but unreliable content.<n>We introduce QIMG-7, a benchmark for multimodal retrieval pollution in multi-sentence factual QA.<n>We propose source-aware trust resolution (SATR), a training-free approach that compares Parametric, Text-only, and Full-MM candidate answers.
  arXiv  Detail & Related papers  (2026-07-12T15:13:09Z)
• SciClaimSeekers at CheckThat! 2026: Retrieving Scientific Sources for Social Media Claims with LLM Reranking [3.969046654861534]
  SciClaimSeekers is a retrieval and reranking framework.<n>System combines BM25 and zero-shot multilingual E5 retrieval with Reciprocal Rank Fusion.<n>Our experiment suggests that large pre-trained models, when combined into a careful pipeline, can be competitive on this task.
  arXiv  Detail & Related papers  (2026-07-06T16:36:46Z)
• Novelty-Aware Agentic Retrieval: Comparing Research Contributions Through Structured Multi-Step Reasoning [0.0]
  Novelty-Aware Research Agent is a prototype agentic retrieval system.<n>It layers structured multi-step reasoning on a RAG pipeline through six typed-contract components.<n>It produces structured comparison artifacts: per-paper contribution records, paper-level overlaps, and a problem x method gap matrix.
  arXiv  Detail & Related papers  (2026-06-20T17:04:02Z)
• How Far Is Document Parsing from Solved? PureDocBench: A Source-TraceableBenchmark across Clean, Degraded, and Real-World Settings [56.70440596502351]
  The past year has seen over 20 open document parsing models, yet benchmarks almost exclusively on OmniDocBench.<n>We present PureDocBench, a benchmark thatrenders document images from HTML/CSS, covering 10 domains, 66 pages, and 1,475 pages.
  arXiv  Detail & Related papers  (2026-05-08T09:30:31Z)
• Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study [0.0]
  This paper presents a systematic empirical comparison of five retrieval strategies within a biomedical question-answering RAG pipeline.<n>All strategies share a fixed generation model (GPT-4o-mini), a common vector store (ChromaDB), and OpenAI's text-em-3-small embeddings.<n> evaluation is conducted on 250 question-answer pairs drawn from a preprocessed subset of the BioASQ benchmark.
  arXiv  Detail & Related papers  (2026-05-04T12:21:46Z)
• vstash: Local-First Hybrid Retrieval with Adaptive Fusion for LLM Agents [0.0]
  We present **vstash**, a local-first document memory system that combines vector similarity search with fulltext keyword matching.<n>All data resides in a single file using sqlite-vec for approximate nearest neighbor search and FTS5 for keyword matching.
  arXiv  Detail & Related papers  (2026-04-16T19:22:58Z)
• Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge [8.680958290253914]
  We present our submission to the LiveRAG Challenge 2025, which evaluates retrieval-augmented generation (RAG) systems on dynamic test sets.<n>Our final hybrid approach combines sparse (BM25) and dense (E5) retrieval methods.<n>We demonstrate that neural re-ranking with RankLLaMA improves MAP from 0.523 to 0.797 but introduces prohibitive computational costs.
  arXiv  Detail & Related papers  (2025-06-27T21:20:43Z)
• Escaping the SpuriVerse: Can Large Vision-Language Models Generalize Beyond Seen Spurious Correlations? [37.703287009808896]
  Finetuning can cause spurious correlations to arise between non-essential features and the target labels.<n>We develop a benchmark by sourcing GPT-4o errors on real-world visual-question-answering (VQA) benchmarks.<n>We evaluate 15 open and closed-source LVLMs on SpuriVerse, finding that even state-of-the-art closed-source models struggle significantly.
  arXiv  Detail & Related papers  (2025-06-23T06:11:43Z)
• Better May Not Be Fairer: A Study on Subgroup Discrepancy in Image Classification [73.87160347728314]
  We investigate how natural background colors play a role as spurious features by annotating the test sets of CIFAR10 and CIFAR100 into subgroups based on the background color of each image. We find that overall human-level accuracy does not guarantee consistent subgroup performances, and the phenomenon remains even on models pre-trained on ImageNet or after data augmentation (DA) Experimental results show that FlowAug achieves more consistent subgroup results than other types of DA methods on CIFAR10/100 and on CIFAR10/100-C.
  arXiv  Detail & Related papers  (2022-12-16T18:51:10Z)
• SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search [40.443716693689794]
  We present a zero-shot ranking algorithm that adapts to COVID-related scientific literature. Despite not relying on TREC-COVID data, our method outperforms models that do.
  arXiv  Detail & Related papers  (2020-10-12T19:28:29Z)
• Automatic sleep stage classification with deep residual networks in a mixed-cohort setting [63.52264764099532]
  We developed a novel deep neural network model to assess the generalizability of several large-scale cohorts. Overall classification accuracy improved with increasing fractions of training data.
  arXiv  Detail & Related papers  (2020-08-21T10:48:35Z)
• CO-Search: COVID-19 Information Retrieval with Semantic Search, Question Answering, and Abstractive Summarization [53.67205506042232]
  CO-Search is a retriever-ranker semantic search engine designed to handle complex queries over the COVID-19 literature. To account for the domain-specific and relatively limited dataset, we generate a bipartite graph of document paragraphs and citations. We evaluate our system on the data of the TREC-COVID information retrieval challenge.
  arXiv  Detail & Related papers  (2020-06-17T01:32:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.