HLTCOE at TREC 2024 NeuCLIR Track

• URL: http://arxiv.org/abs/2510.00143v1
• Date: Tue, 30 Sep 2025 18:16:45 GMT
• Title: HLTCOE at TREC 2024 NeuCLIR Track
• Authors: Eugene Yang, Dawn Lawrie, Orion Weller, James Mayfield,
• Abstract summary: HLT team applied PLAID, an mT5 reranker, GPT-4 reranker, score fusion, and document translation to the TREC 2024 NeuCLIR track.<n>For PLAID we included a variety of models and training techniques -- Translate Distill (TD), Generate Distill (GD) and multi-lingual translate-distill (MTD)
• Score: 23.31564812384347
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The HLTCOE team applied PLAID, an mT5 reranker, GPT-4 reranker, score fusion, and document translation to the TREC 2024 NeuCLIR track. For PLAID we included a variety of models and training techniques -- Translate Distill (TD), Generate Distill (GD) and multi-lingual translate-distill (MTD). TD uses scores from the mT5 model over English MS MARCO query-document pairs to learn how to score query-document pairs where the documents are translated to match the CLIR setting. GD follows TD but uses passages from the collection and queries generated by an LLM for training examples. MTD uses MS MARCO translated into multiple languages, allowing experiments on how to batch the data during training. Finally, for report generation we experimented with system combination over different runs. One family of systems used either GPT-4o or Claude-3.5-Sonnet to summarize the retrieved results from a series of decomposed sub-questions. Another system took the output from those two models and verified/combined them with Claude-3.5-Sonnet. The other family used GPT4o and GPT3.5Turbo to extract and group relevant facts from the retrieved documents based on the decomposed queries. The resulting submissions directly concatenate the grouped facts to form the report and their documents of origin as the citations. The team submitted runs to all NeuCLIR tasks: CLIR and MLIR news tasks as well as the technical documents task and the report generation task.

Related papers

• Retrieval-Augmented Machine Translation with Unstructured Knowledge [63.97706326080482]
  Retrieval-augmented generation (RAG) introduces additional information to enhance large language models (LLMs)<n>In machine translation (MT), previous work typically retrieves in-context examples from paired MT corpora, or domain-specific knowledge from knowledge graphs.<n>In this paper, we study retrieval-augmented MT using unstructured documents.
  arXiv  Detail & Related papers  (2024-12-05T17:00:32Z)
• LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation [67.24113079928668]
  We present LexMatcher, a method for data curation driven by the coverage of senses found in bilingual dictionaries. Our approach outperforms the established baselines on the WMT2022 test sets.
  arXiv  Detail & Related papers  (2024-06-03T15:30:36Z)
• Distillation for Multilingual Information Retrieval [10.223578525761617]
  Translate-Distill framework trains a cross-language neural dual-encoder model using translation and distillation. This work extends Translate-Distill and propose Translate-Distill (MTD) for Multilingual information retrieval. We show that ColBERT-X models trained with MTD outperform their counterparts trained ith Multilingual Translate-Train, by 5% to 25% in nDCG@20 and 15% to 45% in MAP.
  arXiv  Detail & Related papers  (2024-05-02T03:30:03Z)
• MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents [62.02920842630234]
  We show how to build small fact-checking models that have GPT-4-level performance but for 400x lower cost. We do this by constructing synthetic training data with GPT-4, which involves creating realistic yet challenging instances of factual errors. For evaluation, we unify datasets from recent work on fact-checking and grounding LLM generations into a new benchmark, LLM-AggreFact.
  arXiv  Detail & Related papers  (2024-04-16T17:59:10Z)
• HLTCOE at TREC 2023 NeuCLIR Track [10.223578525761617]
  The HLT team applied PLAID, an mT5 reranker, and document translation to the TREC 2023 NeuCLIR track. For PLAID we included a variety of models and training techniques -- the English model released with ColBERT v2, translate-train(TT), Translate Distill(TD) and translate multilingual-train(MTT)
  arXiv  Detail & Related papers  (2024-04-11T20:46:18Z)
• PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion [96.47420221442397]
  We introduce the PowerPoint Task Completion benchmark to assess the ability of Large Language Models to finish multi-turn, multi-modal instructions. We also propose the PPTX-Match Evaluation System that evaluates if LLMs finish the instruction based on the prediction file rather than the label API sequence. The results show that GPT-4 outperforms other LLMs with 75.1% accuracy in single-turn dialogue testing but faces challenges in completing entire sessions, achieving just 6% session accuracy.
  arXiv  Detail & Related papers  (2023-11-03T08:06:35Z)
• Data-to-text Generation for Severely Under-Resourced Languages with GPT-3.5: A Bit of Help Needed from Google Translate [5.632410663467911]
  We look at how language learning systems cope with tasks involving languages that are severely under-represented in their training data. This includes data-to-text generation for Irish, Maltese, Welsh and Breton. We find that few-shot prompting works better for direct generation into under-resourced languages, but that the difference disappears when pivoting via English. We conclude that good performance on under-resourced languages can be achieved out-of-the box with state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2023-08-19T09:19:34Z)
• Document-Level Machine Translation with Large Language Models [91.03359121149595]
  Large language models (LLMs) can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks. This paper provides an in-depth evaluation of LLMs' ability on discourse modeling.
  arXiv  Detail & Related papers  (2023-04-05T03:49:06Z)
• Multilingual ColBERT-X [11.768656900939048]
  ColBERT-X is a dense retrieval model for Cross Language Information Retrieval ( CLIR) In CLIR, documents are written in one natural language, while the queries are expressed in another. A related task is multilingual IR (MLIR) where the system creates a single ranked list of documents written in many languages.
  arXiv  Detail & Related papers  (2022-09-03T06:02:52Z)
• Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task [95.06453182273027]
  This report describes Microsoft's machine translation systems for the WMT21 shared task on large-scale multilingual machine translation. Our model submissions to the shared task were with DeltaLMnotefooturlhttps://aka.ms/deltalm, a generic pre-trained multilingual-decoder model. Our final submissions ranked first on three tracks in terms of the automatic evaluation metric.
  arXiv  Detail & Related papers  (2021-11-03T09:16:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.