Top-P Masking for Cross Language Information Retrieval

• URL: http://arxiv.org/abs/2510.19758v1
• Date: Wed, 22 Oct 2025 16:47:42 GMT
• Title: Top-P Masking for Cross Language Information Retrieval
• Authors: Joseph Casale, Andrew Silverschotz, Joseph DeSimone,
• Abstract summary: We propose using Top-P Dynamic Masking to promote sparse representations in Information Retrieval tasks.<n>Specifically, we evaluate our methods in the domain of Cross Language Information Retrieval (CLIR)
• Score: 0.0
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Top-K masking schemes have been proposed as a method to promote sparse representations in Information Retrieval (IR) tasks, as a simple alternative to Floating Point Operations per Second (FLOPS) regularization. Algorithms such as Bilingual Lexical and Document Expansion Model (BLADE), adopt this approach as a post-processing stage. We propose using Top-P Dynamic Masking similar to Nucleus Sampling in Large Language Models, and demonstrate better performance than Top-K masking. Specifically, we evaluate our methods in the domain of Cross Language Information Retrieval (CLIR)

Related papers

• Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection [54.21851618853518]
  We present a concise yet effective approach called Patch Generation-to-Selection to enhance CLIP's training efficiency.<n>Our approach, CLIP-PGS, sets new state-of-the-art results in zero-shot classification and retrieval tasks.
  arXiv  Detail & Related papers  (2025-03-21T12:10:38Z)
• Adapting Vision-Language Model with Fine-grained Semantics for Open-Vocabulary Segmentation [42.020470627552136]
  Open-vocabulary segmentation is primarily bottlenecked by mask classification, not mask generation.<n>We propose a novel Fine-grained Semantic Adaptation (FISA) method to address this limitation.<n>FISA enhances the extracted visual features with fine-grained semantic awareness by explicitly integrating this crucial semantic information early in the visual encoding process.
  arXiv  Detail & Related papers  (2024-09-24T17:50:28Z)
• MaskInversion: Localized Embeddings via Optimization of Explainability Maps [49.50785637749757]
  MaskInversion generates a context-aware embedding for a query image region specified by a mask at test time. It can be used for a broad range of tasks, including open-vocabulary class retrieval, referring expression comprehension, as well as for localized captioning and image generation.
  arXiv  Detail & Related papers  (2024-07-29T14:21:07Z)
• PosSAM: Panoptic Open-vocabulary Segment Anything [58.72494640363136]
  PosSAM is an open-vocabulary panoptic segmentation model that unifies the strengths of the Segment Anything Model (SAM) with the vision-native CLIP model in an end-to-end framework. We introduce a Mask-Aware Selective Ensembling (MASE) algorithm that adaptively enhances the quality of generated masks and boosts the performance of open-vocabulary classification during inference for each image.
  arXiv  Detail & Related papers  (2024-03-14T17:55:03Z)
• Iterative Mask Filling: An Effective Text Augmentation Method Using Masked Language Modeling [0.0]
  We propose a novel text augmentation method that leverages the Fill-Mask feature of the transformer-based BERT model. Our method involves iteratively masking words in a sentence and replacing them with language model predictions. Experimental results show that our proposed method significantly improves performance, especially on topic classification datasets.
  arXiv  Detail & Related papers  (2024-01-03T16:47:13Z)
• DFormer: Diffusion-guided Transformer for Universal Image Segmentation [86.73405604947459]
  The proposed DFormer views universal image segmentation task as a denoising process using a diffusion model. At inference, our DFormer directly predicts the masks and corresponding categories from a set of randomly-generated masks. Our DFormer outperforms the recent diffusion-based panoptic segmentation method Pix2Seq-D with a gain of 3.6% on MS COCO val 2017 set.
  arXiv  Detail & Related papers  (2023-06-06T06:33:32Z)
• Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval [80.43859162884353]
  We propose a multilingual multilingual language model called masked sentence model (MSM)<n>MSM consists of a sentence encoder to generate the sentence representations, and a document encoder applied to a sequence of sentence vectors from a document.<n>To train the model, we propose a masked sentence prediction task, which masks and predicts the sentence vector via a hierarchical contrastive loss with sampled negatives.
  arXiv  Detail & Related papers  (2023-02-03T09:54:27Z)
• Retrieval Oriented Masking Pre-training Language Model for Dense Passage Retrieval [16.592276887533714]
  Masked Language Modeling (MLM) is a major sub-task of the pre-training process. Traditional random masking strategy tend to select a large number of tokens that have limited effect on the passage retrieval task. We propose alternative retrieval oriented masking (dubbed as ROM) strategy where more important tokens will have a higher probability of being masked out.
  arXiv  Detail & Related papers  (2022-10-27T02:43:48Z)
• InforMask: Unsupervised Informative Masking for Language Model Pretraining [13.177839395411858]
  We propose a new unsupervised masking strategy for training masked language models. InforMask exploits Pointwise Mutual Information (PMI) to select the most informative tokens to mask.
  arXiv  Detail & Related papers  (2022-10-21T07:10:56Z)
• Masking as an Efficient Alternative to Finetuning for Pretrained Language Models [49.64561153284428]
  We learn selective binary masks for pretrained weights in lieu of modifying them through finetuning. In intrinsic evaluations, we show that representations computed by masked language models encode information necessary for solving downstream tasks.
  arXiv  Detail & Related papers  (2020-04-26T15:03:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.