PASH at TREC 2021 Deep Learning Track: Generative Enhanced Model for Multi-stage Ranking

• URL: http://arxiv.org/abs/2205.11245v4
• Date: Wed, 28 Aug 2024 08:51:57 GMT
• Title: PASH at TREC 2021 Deep Learning Track: Generative Enhanced Model for Multi-stage Ranking
• Authors: Yixuan Qiao, Hao Chen, Jun Wang, Tuozhen Liu, Xianbin Ye, Xin Tang, Rui Fang, Peng Gao, Wenfeng Xie, Guotong Xie,
• Abstract summary: This paper describes the PASH participation in TREC 2021 Deep Learning Track. In the recall stage, we adopt a scheme combining sparse and dense retrieval method. In the multi-stage ranking phase, point-wise and pair-wise ranking strategies are used.
• Score: 20.260222175405215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper describes the PASH participation in TREC 2021 Deep Learning Track. In the recall stage, we adopt a scheme combining sparse and dense retrieval method. In the multi-stage ranking phase, point-wise and pair-wise ranking strategies are used one after another based on model continual pre-trained on general knowledge and document-level data. Compared to TREC 2020 Deep Learning Track, we have additionally introduced the generative model T5 to further enhance the performance.

Related papers

• SmurfCat at SemEval-2024 Task 6: Leveraging Synthetic Data for Hallucination Detection [51.99159169107426]
  We present our novel systems developed for the SemEval-2024 hallucination detection task. Our investigation spans a range of strategies to compare model predictions with reference standards. We introduce three distinct methods that exhibit strong performance metrics.
  arXiv  Detail & Related papers  (2024-04-09T09:03:44Z)
• InternLM2 Technical Report [159.70692271378581]
  This paper introduces InternLM2, an open-source Large Language Models (LLMs) that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks. The pre-training process of InternLM2 is meticulously detailed, highlighting the preparation of diverse data types. InternLM2 efficiently captures long-term dependencies, initially trained on 4k tokens before advancing to 32k tokens in pre-training and fine-tuning stages.
  arXiv  Detail & Related papers  (2024-03-26T00:53:24Z)
• Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
  Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones. This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
  arXiv  Detail & Related papers  (2024-01-29T18:27:52Z)
• When Parameter-efficient Tuning Meets General-purpose Vision-language Models [65.19127815275307]
  PETAL revolutionizes the training process by requiring only 0.5% of the total parameters, achieved through a unique mode approximation technique. Our experiments reveal that PETAL not only outperforms current state-of-the-art methods in most scenarios but also surpasses full fine-tuning models in effectiveness.
  arXiv  Detail & Related papers  (2023-12-16T17:13:08Z)
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [71.63186089279218]
  This paper introduces a pre-trained model-based continual learning toolbox known as PILOT. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
  arXiv  Detail & Related papers  (2023-09-13T17:55:11Z)
• Hybrid Retrieval and Multi-stage Text Ranking Solution at TREC 2022 Deep Learning Track [22.81602641419962]
  We explain the hybrid text retrieval and multi-stage text ranking method adopted in our solution. In the ranking stage, in addition to the full interaction-based ranking model built on large pre-trained language model, we also proposes a lightweight sub-ranking module. Our models achieve the 1st and 4th rank on the test set of passage ranking and document ranking respectively.
  arXiv  Detail & Related papers  (2023-08-23T09:56:59Z)
• Progressive Prompts: Continual Learning for Language Models [38.80713056417491]
  We introduce Progressive Prompts - a simple and efficient approach for continual learning in language models. Progressive Prompts learns a new soft prompt for each task and sequentially resists it with the previously learned prompts. Experiments on standard continual learning benchmarks show that our approach outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-01-29T00:17:38Z)
• Predicting Query-Item Relationship using Adversarial Training and Robust Modeling Techniques [2.4442957793630584]
  We present an effective way to predict search query-item relationship. We combine pre-trained transformer and LSTM models, and increase model robustness using adversarial training. Applying our strategies, we achieved 10th place in KDD Cup 2022 Product Substitution Classification task.
  arXiv  Detail & Related papers  (2022-08-23T06:10:06Z)
• Unifying Language Learning Paradigms [96.35981503087567]
  We present a unified framework for pre-training models that are universally effective across datasets and setups. We show how different pre-training objectives can be cast as one another and how interpolating between different objectives can be effective. Our model also achieve strong results at in-context learning, outperforming 175B GPT-3 on zero-shot SuperGLUE and tripling the performance of T5-XXL on one-shot summarization.
  arXiv  Detail & Related papers  (2022-05-10T19:32:20Z)
• Overview of the TREC 2020 deep learning track [30.531644711518414]
  This year we have a document retrieval task and a passage retrieval task, each with hundreds of thousands of human-labeled training queries. We evaluate using single-shot TREC-style evaluation, to give us a picture of which ranking methods work best when large data is available. This year we have further evidence that rankers with BERT-style pretraining outperform other rankers in the large data regime.
  arXiv  Detail & Related papers  (2021-02-15T16:47:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.