FecTek: Enhancing Term Weight in Lexicon-Based Retrieval with Feature Context and Term-level Knowledge

• URL: http://arxiv.org/abs/2404.12152v1
• Date: Thu, 18 Apr 2024 12:58:36 GMT
• Title: FecTek: Enhancing Term Weight in Lexicon-Based Retrieval with Feature Context and Term-level Knowledge
• Authors: Zunran Wang, Zhonghua Li, Wei Shen, Qi Ye, Liqiang Nie,
• Abstract summary: We introduce an innovative method by introducing FEature Context and TErm-level Knowledge modules. To effectively enrich the feature context representations of term weight, the Feature Context Module (FCM) is introduced. We also develop a term-level knowledge guidance module (TKGM) for effectively utilizing term-level knowledge to intelligently guide the modeling process of term weight.
• Score: 54.61068946420894
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lexicon-based retrieval has gained siginificant popularity in text retrieval due to its efficient and robust performance. To further enhance performance of lexicon-based retrieval, researchers have been diligently incorporating state-of-the-art methodologies like Neural retrieval and text-level contrastive learning approaches. Nonetheless, despite the promising outcomes, current lexicon-based retrieval methods have received limited attention in exploring the potential benefits of feature context representations and term-level knowledge guidance. In this paper, we introduce an innovative method by introducing FEature Context and TErm-level Knowledge modules(FecTek). To effectively enrich the feature context representations of term weight, the Feature Context Module (FCM) is introduced, which leverages the power of BERT's representation to determine dynamic weights for each element in the embedding. Additionally, we develop a term-level knowledge guidance module (TKGM) for effectively utilizing term-level knowledge to intelligently guide the modeling process of term weight. Evaluation of the proposed method on MS Marco benchmark demonstrates its superiority over the previous state-of-the-art approaches.

Related papers

• Preserving Generalization of Language models in Few-shot Continual Relation Extraction [34.68364639170838]
  Few-shot Continual Relations Extraction (FCRE) is an emerging and dynamic area of study. We introduce a novel method that leverages often-discarded language model heads. Our experimental results underscore the efficacy of the proposed method and offer valuable insights for future work.
  arXiv  Detail & Related papers  (2024-10-01T02:22:34Z)
• HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modeling [39.14392943549792]
  We propose a novel approach called Hierarchical Prompt Tuning (HPT), enabling simultaneous modeling of both structured and conventional linguistic knowledge. We introduce a relationship-guided attention module to capture pair-wise associations among entities and attributes for low-level prompt learning. By incorporating high-level and global-level prompts modeling overall semantics, the proposed hierarchical structure forges cross-level interlinks and empowers the model to handle more complex and long-term relationships.
  arXiv  Detail & Related papers  (2024-08-27T06:50:28Z)
• Knowledge Graphs and Pre-trained Language Models enhanced Representation Learning for Conversational Recommender Systems [58.561904356651276]
  We introduce the Knowledge-Enhanced Entity Representation Learning (KERL) framework to improve the semantic understanding of entities for Conversational recommender systems. KERL uses a knowledge graph and a pre-trained language model to improve the semantic understanding of entities. KERL achieves state-of-the-art results in both recommendation and response generation tasks.
  arXiv  Detail & Related papers  (2023-12-18T06:41:23Z)
• Unifying Structure and Language Semantic for Efficient Contrastive Knowledge Graph Completion with Structured Entity Anchors [0.3913403111891026]
  The goal of knowledge graph completion (KGC) is to predict missing links in a KG using trained facts that are already known. We propose a novel method to effectively unify structure information and language semantics without losing the power of inductive reasoning.
  arXiv  Detail & Related papers  (2023-11-07T11:17:55Z)
• Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning [63.148199057487226]
  We propose a modular, NEuroSymbolic Textual Agent (NESTA) that combines a generic semantic generalization with a rule induction system to learn interpretable rules as policies. Our experiments show that the proposed NESTA method outperforms deep reinforcement learning-based techniques by achieving better to unseen test games and learning from fewer training interactions.
  arXiv  Detail & Related papers  (2023-07-05T23:21:05Z)
• Evolving Knowledge Mining for Class Incremental Segmentation [113.59611699693092]
  Class Incremental Semantic (CISS) has been a trend recently due to its great significance in real-world applications. We propose a novel method, Evolving kNowleDge minING, employing a frozen backbone. We evaluate our method on two widely used benchmarks and consistently demonstrate new state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-06-03T07:03:15Z)
• Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis [96.53859361560505]
  We propose a knowledge graph augmented network (KGAN) to incorporate external knowledge with explicitly syntactic and contextual information. KGAN captures the sentiment feature representations from multiple perspectives, i.e., context-, syntax- and knowledge-based. Experiments on three popular ABSA benchmarks demonstrate the effectiveness and robustness of our KGAN.
  arXiv  Detail & Related papers  (2022-01-13T08:25:53Z)
• KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs [26.557447199727758]
  We propose a novel knowledge-aware language model framework based on fine-tuning process. Our model can efficiently incorporate world knowledge from KGs into existing language models such as BERT.
  arXiv  Detail & Related papers  (2021-09-09T12:39:17Z)
• Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources [26.256653692882715]
  How to engage commonsense effectively in question answering systems is still under exploration. We propose a novel question-answering method by integrating ConceptNet, Wikipedia, and the Cambridge Dictionary. We use a pre-trained language model to encode the question, retrieved knowledge and choices, and propose an answer choice-aware attention mechanism.
  arXiv  Detail & Related papers  (2020-11-05T08:50:43Z)
• CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models [103.18329049830152]
  We propose a novel framework named Coke to dynamically select contextual knowledge and embed knowledge context according to textual context. Our experimental results show that Coke outperforms various baselines on typical knowledge-driven NLP tasks. Coke can describe the semantics of text-related knowledge in a more interpretable form than the conventional PLMs.
  arXiv  Detail & Related papers  (2020-09-29T12:29:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.