Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time

• URL: http://arxiv.org/abs/2210.09179v1
• Date: Mon, 17 Oct 2022 15:28:54 GMT
• Title: Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time
• Abstract summary: We use Transformer Language Models to get entailment probabilities and investigate different types of queries. We show that one can reduce the close reading time by taking some percentage of the ranked documents.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We approach the classification problem as an entailment problem and apply zero-shot ranking to socio-political texts. Documents that are ranked at the top can be considered positively classified documents and this reduces the close reading time for the information extraction process. We use Transformer Language Models to get the entailment probabilities and investigate different types of queries. We find that DeBERTa achieves higher mean average precision scores than RoBERTa and when declarative form of the class label is used as a query, it outperforms dictionary definition of the class label. We show that one can reduce the close reading time by taking some percentage of the ranked documents that the percentage depends on how much recall they want to achieve. However, our findings also show that percentage of the documents that should be read increases as the topic gets broader.

Related papers

• ReLeVAnT: Relevance Lexical Vectors for Accurate Legal Text Classification [5.530730417910086]
  ReLeVAnT is a framework for legal document binary classification.<n>It uses n-gram processing, contrastive score matching, and a shallow neural network as the primary drivers for discriminative classification.<n>It has 99.3% accuracy and 98.7% F1 score on the LexGLUE dataset.
  arXiv  Detail & Related papers  (2026-04-24T07:13:24Z)
• Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Ranking [2.584263027095689]
  Transformer-based models like BERT excel at short text classification but struggle with long document classification (LDC)<n>We propose an efficient, zero-shot approach to LDC that leverages sentence ranking to reduce input context without altering the model architecture.<n>Our method enables the adaptation of models trained on short texts, such as headlines, to long-form documents by selecting the most informative sentences.
  arXiv  Detail & Related papers  (2025-08-24T18:52:37Z)
• Evaluating D-MERIT of Partial-annotation on Information Retrieval [77.44452769932676]
  Retrieval models are often evaluated on partially-annotated datasets. We show that using partially-annotated datasets in evaluation can paint a distorted picture.
  arXiv  Detail & Related papers  (2024-06-23T08:24:08Z)
• Mitigating Word Bias in Zero-shot Prompt-based Classifiers [55.60306377044225]
  We show that matching class priors correlates strongly with the oracle upper bound performance. We also demonstrate large consistent performance gains for prompt settings over a range of NLP tasks.
  arXiv  Detail & Related papers  (2023-09-10T10:57:41Z)
• Contextual Biasing of Named-Entities with Large Language Models [12.396054621526643]
  This paper studies contextual biasing with Large Language Models (LLMs) During second-pass rescoring additional contextual information is provided to a LLM to boost Automatic Speech Recognition (ASR) performance. We propose to leverage prompts for a LLM without fine tuning during rescoring which incorporate a biasing list and few-shot examples.
  arXiv  Detail & Related papers  (2023-09-01T20:15:48Z)
• M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [103.6153593636399]
  We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning) It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario. Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
  arXiv  Detail & Related papers  (2023-03-09T09:05:47Z)
• Query Expansion Using Contextual Clue Sampling with Language Models [69.51976926838232]
  We propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context. Our lexical matching based approach achieves a similar top-5/top-20 retrieval accuracy and higher top-100 accuracy compared with the well-established dense retrieval model DPR. For end-to-end QA, the reader model also benefits from our method and achieves the highest Exact-Match score against several competitive baselines.
  arXiv  Detail & Related papers  (2022-10-13T15:18:04Z)
• None Class Ranking Loss for Document-Level Relation Extraction [22.173080823450498]
  Document-level relation extraction (RE) aims at extracting relations among entities expressed across multiple sentences. In a typical document, most entity pairs do not express any pre-defined relation and are labeled as "none" or "no relation"
  arXiv  Detail & Related papers  (2022-05-01T14:24:37Z)
• Improving Probabilistic Models in Text Classification via Active Learning [0.0]
  We propose a fast new model for text classification that combines information from both labeled and unlabeled data with an active learning component. We show that by introducing information about the structure of unlabeled data and iteratively labeling uncertain documents, our model improves performance.
  arXiv  Detail & Related papers  (2022-02-05T20:09:26Z)
• ZeroBERTo -- Leveraging Zero-Shot Text Classification by Topic Modeling [57.80052276304937]
  This paper proposes a new model, ZeroBERTo, which leverages an unsupervised clustering step to obtain a compressed data representation before the classification task. We show that ZeroBERTo has better performance for long inputs and shorter execution time, outperforming XLM-R by about 12% in the F1 score in the FolhaUOL dataset.
  arXiv  Detail & Related papers  (2022-01-04T20:08:17Z)
• LeQua@CLEF2022: Learning to Quantify [76.22817970624875]
  LeQua 2022 is a new lab for the evaluation of methods for learning to quantify'' in textual datasets. The goal of this lab is to provide a setting for the comparative evaluation of methods for learning to quantify, both in the binary setting and in the single-label multiclass setting.
  arXiv  Detail & Related papers  (2021-11-22T14:54:20Z)
• Rank over Class: The Untapped Potential of Ranking in Natural Language Processing [8.637110868126546]
  We argue that many tasks which are currently addressed using classification are in fact being shoehorned into a classification mould. We propose a novel end-to-end ranking approach consisting of a Transformer network responsible for producing representations for a pair of text sequences. In an experiment on a heavily-skewed sentiment analysis dataset, converting ranking results to classification labels yields an approximately 22% improvement over state-of-the-art text classification.
  arXiv  Detail & Related papers  (2020-09-10T22:18:57Z)
• Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question Answering [9.480648914353035]
  We present FiRA: a novel dataset of Fine-Grained Relevances. We extend the ranked retrieval annotations of the Deep Learning track of TREC 2019 with passage and word level graded relevance annotations for all relevant documents. As an example, we evaluate the recently introduced TKL document ranking model. We find that although TKL exhibits state-of-the-art retrieval results for long documents, it misses many relevant passages.
  arXiv  Detail & Related papers  (2020-08-12T14:59:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.