Overview of the TREC 2025 Tip-of-the-Tongue track

• URL: http://arxiv.org/abs/2601.20671v1
• Date: Wed, 28 Jan 2026 14:51:52 GMT
• Title: Overview of the TREC 2025 Tip-of-the-Tongue track
• Abstract summary: The TREC 2025 ToT track focused on a single ad-hoc retrieval task.<n>This year, we extended the track to general domain and incorporated different sets of test queries from diverse sources.
• Score: 44.18900757791221
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tip-of-the-tongue (ToT) known-item retrieval involves re-finding an item for which the searcher does not reliably recall an identifier. ToT information requests (or queries) are verbose and tend to include several complex phenomena, making them especially difficult for existing information retrieval systems. The TREC 2025 ToT track focused on a single ad-hoc retrieval task. This year, we extended the track to general domain and incorporated different sets of test queries from diverse sources, namely from the MS-ToT dataset, manual topic development, and LLM-based synthetic query generation. This year, 9 groups (including the track coordinators) submitted 32 runs.

Related papers

• Overview of the TREC 2025 Retrieval Augmented Generation (RAG) Track [52.136178637532886]
  Second edition of the TREC Retrieval Augmented Generation (RAG) Track advances research on systems that integrate retrieval and generation.<n>Building on the foundation of the inaugural 2024 track, this year's challenge introduces long, multi-sentence narrative queries.<n>Participants are tasked with designing pipelines that combine retrieval and generation while ensuring transparency and factual grounding.
  arXiv  Detail & Related papers  (2026-03-10T16:49:18Z)
• GISA: A Benchmark for General Information-Seeking Assistant [102.30831921333755]
  GISA is a benchmark for General Information-Seeking Assistants comprising 373 human-crafted queries.<n>It integrates both deep reasoning and broad information aggregation within unified tasks, and includes a live subset with periodically updated answers to resist memorization.<n>Experiments on mainstream LLMs and commercial search products reveal that even the best-performing model achieves only 19.30% exact match score.
  arXiv  Detail & Related papers  (2026-02-09T11:44:15Z)
• Tip of the Tongue Query Elicitation for Simulated Evaluation [49.083870362622164]
  Tip-of-the-tongue (TOT) search occurs when a user struggles to recall a specific identifier, such as a document title.<n>Current approaches rely heavily on community question-answering (CQA) websites, leading to labor-intensive evaluation and domain bias.<n>We introduce two methods for eliciting TOT queries - leveraging large language models (LLMs) and human participants.
  arXiv  Detail & Related papers  (2025-02-25T02:11:42Z)
• Holistically Guided Monte Carlo Tree Search for Intricate Information Seeking [118.3983437282541]
  We introduce an LLM-based search assistant that adopts a new information seeking paradigm with holistically guided Monte Carlo tree search (HG-MCTS)<n>We reformulate the task as a progressive information collection process with a knowledge memory and unite an adaptive checklist with multi-perspective reward modeling in MCTS.<n>Our multi-perspective reward modeling offers both exploration and retrieval rewards, along with progress feedback that tracks completed and remaining sub-goals.
  arXiv  Detail & Related papers  (2025-02-07T08:36:39Z)
• RALI@TREC iKAT 2024: Achieving Personalization via Retrieval Fusion in Conversational Search [24.464088639113417]
  In personalized search, effectively capturing a user's complex search intent requires incorporating both contextual information and key elements from the user profile into query reformulation.<n>This is a challenge we denote as over-personalization.<n>To address this, we propose different strategies by fusing ranking lists generated from the queries with different levels of personalization.
  arXiv  Detail & Related papers  (2024-12-11T00:44:52Z)
• Multi-hop Evidence Pursuit Meets the Web: Team Papelo at FEVER 2024 [1.3923460621808879]
  We show that the reasoning power of large language models (LLMs) and the retrieval power of modern search engines can be combined to automate this process. We integrate LLMs and search under a multi-hop evidence pursuit strategy. Our submitted system achieves.510 AVeriTeC score on the dev set and.477 AVeriTeC score on the test set.
  arXiv  Detail & Related papers  (2024-11-08T18:25:06Z)
• I3: Intent-Introspective Retrieval Conditioned on Instructions [83.91776238599824]
  I3 is a unified retrieval system that performs Intent-Introspective retrieval across various tasks conditioned on Instructions without task-specific training. I3 incorporates a pluggable introspector in a parameter-isolated manner to comprehend specific retrieval intents. It utilizes extensive LLM-generated data to train I3 phase-by-phase, embodying two key designs: progressive structure pruning and drawback-based data refinement.
  arXiv  Detail & Related papers  (2023-08-19T14:17:57Z)
• Decomposing Complex Queries for Tip-of-the-tongue Retrieval [72.07449449115167]
  Complex queries describe content elements (e.g., book characters or events), information beyond the document text. This retrieval setting, called tip of the tongue (TOT), is especially challenging for models reliant on lexical and semantic overlap between query and document text. We introduce a simple yet effective framework for handling such complex queries by decomposing the query into individual clues, routing those as sub-queries to specialized retrievers, and ensembling the results.
  arXiv  Detail & Related papers  (2023-05-24T11:43:40Z)
• Leveraging Query Resolution and Reading Comprehension for Conversational Passage Retrieval [6.490148466525755]
  This paper describes the participation of UvA.ILPS group at the TREC CAsT 2020 track. Our pipeline consists of (i) an initial retrieval module that uses BM25, and (ii) a re-ranking module that combines the score of a BERT ranking model with the score of a machine comprehension model adjusted for passage retrieval.
  arXiv  Detail & Related papers  (2021-02-17T14:41:57Z)
• Query Resolution for Conversational Search with Limited Supervision [63.131221660019776]
  We propose QuReTeC (Query Resolution by Term Classification), a neural query resolution model based on bidirectional transformers. We show that QuReTeC outperforms state-of-the-art models, and furthermore, that our distant supervision method can be used to substantially reduce the amount of human-curated data required to train QuReTeC.
  arXiv  Detail & Related papers  (2020-05-24T11:37:22Z)
• TREC CAsT 2019: The Conversational Assistance Track Overview [34.65827453762031]
  The Conversational Assistance Track (CAsT) is a new track for TREC 2019 to facilitate Conversational Information Seeking (CIS) research. The document corpus is 38,426,252 passages from the TREC Complex Answer Retrieval (CAR) and Microsoft MAchine Reading COmprehension (MARCO) datasets. This year 21 groups submitted a total of 65 runs using varying methods for conversational query understanding and ranking.
  arXiv  Detail & Related papers  (2020-03-30T16:58:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.