RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble

• URL: http://arxiv.org/abs/2010.00200v1
• Date: Thu, 1 Oct 2020 05:27:51 GMT
• Title: RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble
• Authors: Michael Bendersky and Honglei Zhuang and Ji Ma and Shuguang Han and Keith Hall and Ryan McDonald
• Abstract summary: We propose a weighted hierarchical rank fusion approach to meet the challenge of building a search engine for rapidly evolving biomedical collection. Our ablation studies demonstrate the contributions of each of these systems to the overall ensemble. The submitted ensemble runs achieved state-of-the-art performance in rounds 4 and 5 of the TREC-COVID challenge.
• Score: 19.041809003928506
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we report the results of our participation in the TREC-COVID challenge. To meet the challenge of building a search engine for rapidly evolving biomedical collection, we propose a simple yet effective weighted hierarchical rank fusion approach, that ensembles together 102 runs from (a) lexical and semantic retrieval systems, (b) pre-trained and fine-tuned BERT rankers, and (c) relevance feedback runs. Our ablation studies demonstrate the contributions of each of these systems to the overall ensemble. The submitted ensemble runs achieved state-of-the-art performance in rounds 4 and 5 of the TREC-COVID challenge.

Related papers

• Summary of the NOTSOFAR-1 Challenge: Highlights and Learnings [14.045317709780313]
  The first Natural Office Talkers in Settings of Far-field Audio Recordings (NOTSOFAR-1) Challenge is a pivotal initiative that sets new benchmarks. The challenge provides a unique combination of 280 recorded meetings across 30 diverse environments, capturing real-world acoustic conditions and conversational dynamics. We provide an overview of the systems submitted to the challenge and analyze the top-performing approaches.
  arXiv  Detail & Related papers  (2025-01-28T21:25:08Z)
• Chain-of-Retrieval Augmented Generation [72.06205327186069]
  This paper introduces an approach for training o1-like RAG models that retrieve and reason over relevant information step by step before generating the final answer. Our proposed method, CoRAG, allows the model to dynamically reformulate the query based on the evolving state.
  arXiv  Detail & Related papers  (2025-01-24T09:12:52Z)
• Overview of AI-Debater 2023: The Challenges of Argument Generation Tasks [62.443665295250035]
  We present the results of the AI-Debater 2023 Challenge held by the Chinese Conference on Affect Computing (CCAC 2023) In total, 32 competing teams register for the challenge, from which we received 11 successful submissions.
  arXiv  Detail & Related papers  (2024-07-20T10:13:54Z)
• Towards Unsupervised Question Answering System with Multi-level Summarization for Legal Text [0.0]
  This paper summarizes Team SCaLAR's work on SemEval-2024 Task 5: Legal Argument Reasoning in Civil Procedure. We propose a simple yet novel similarity and distance-based unsupervised approach to generate labels. Our unsupervised system witnessed a 20-point increase in macro F1-score on the development set and a 10-point increase on the test set.
  arXiv  Detail & Related papers  (2024-03-19T19:15:13Z)
• CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models [8.237131071390715]
  We consider the challenge of summarizing patients' medical progress notes in a limited data setting. For the Problem List Summarization (shared task 1A) at the BioNLP Workshop 2023, we demonstrate that Clinical-T5 fine-tuned to 765 medical clinic notes outperforms other extractive, abstractive and zero-shot baselines.
  arXiv  Detail & Related papers  (2023-06-08T16:08:10Z)
• Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking [83.40120598637665]
  Zero-shot transfer learning for Dialogue State Tracking (DST) helps to handle a variety of task-oriented dialogue domains without the cost of collecting in-domain data. Existing works mainly study common data- or model-level augmentation methods to enhance the generalization. We present a simple and effective "divide, conquer and combine" solution, which explicitly disentangles the semantics of seen data.
  arXiv  Detail & Related papers  (2023-06-01T08:21:20Z)
• ICDAR 2023 Competition on Hierarchical Text Detection and Recognition [60.68100769639923]
  The competition is aimed to promote research into deep learning models and systems that can jointly perform text detection and recognition. We present details of the proposed competition organization, including tasks, datasets, evaluations, and schedule. During the competition period (from January 2nd 2023 to April 1st 2023), at least 50 submissions from more than 20 teams were made in the 2 proposed tasks.
  arXiv  Detail & Related papers  (2023-05-16T18:56:12Z)
• COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization [84.70895015194188]
  We propose a Contrastive Learning based re-ranking framework for one-stage summarization called COLO. COLO boosts the extractive and abstractive results of one-stage systems on CNN/DailyMail benchmark to 44.58 and 46.33 ROUGE-1 score.
  arXiv  Detail & Related papers  (2022-09-29T06:11:21Z)
• Rotation Invariance and Extensive Data Augmentation: a strategy for the Mitosis Domain Generalization (MIDOG) Challenge [1.52292571922932]
  We present the strategy we applied to participate in the MIDOG 2021 competition. The purpose of the competition was to evaluate the generalization of solutions to images acquired with unseen target scanners. We propose a solution based on a combination of state-of-the-art deep learning methods.
  arXiv  Detail & Related papers  (2021-09-02T10:09:02Z)
• NTIRE 2021 Multi-modal Aerial View Object Classification Challenge [88.89190054948325]
  We introduce the first Challenge on Multi-modal Aerial View Object Classification (MAVOC) in conjunction with the NTIRE 2021 workshop at CVPR. This challenge is composed of two different tracks using EO and SAR imagery. We discuss the top methods submitted for this competition and evaluate their results on our blind test set.
  arXiv  Detail & Related papers  (2021-07-02T16:55:08Z)
• Overview of BioASQ 2021: The ninth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering [0.293168019422713]
  The BioASQ challenge aims to advance the state-of-the-art in large-scale biomedical semantic indexing and question answering. This paper presents an overview of the ninth edition of the BioASQ challenge in the context of the Conference and Labs of the Evaluation Forum (CLEF) 2021. In total, 42 teams with more than 170 systems were registered to participate in the four tasks of the challenge.
  arXiv  Detail & Related papers  (2021-06-28T10:03:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.