Towards Detection of Subjective Bias using Contextualized Word Embeddings

• URL: http://arxiv.org/abs/2002.06644v1
• Date: Sun, 16 Feb 2020 18:39:16 GMT
• Title: Towards Detection of Subjective Bias using Contextualized Word Embeddings
• Authors: Tanvi Dadu, Kartikey Pant and Radhika Mamidi
• Abstract summary: We perform experiments for detecting subjective bias using BERT-based models on the Wiki Neutrality Corpus(WNC) The dataset consists of $360k$ labeled instances, from Wikipedia edits that remove various instances of the bias.
• Score: 9.475039534437332
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Subjective bias detection is critical for applications like propaganda detection, content recommendation, sentiment analysis, and bias neutralization. This bias is introduced in natural language via inflammatory words and phrases, casting doubt over facts, and presupposing the truth. In this work, we perform comprehensive experiments for detecting subjective bias using BERT-based models on the Wiki Neutrality Corpus(WNC). The dataset consists of $360k$ labeled instances, from Wikipedia edits that remove various instances of the bias. We further propose BERT-based ensembles that outperform state-of-the-art methods like $BERT_{large}$ by a margin of $5.6$ F1 score.

Related papers

• Eliminating the Language Bias for Visual Question Answering with fine-grained Causal Intervention [9.859335795616028]
  We propose a novel causal intervention training scheme named CIBi to eliminate language bias from a finer-grained perspective. We employ causal intervention and contrastive learning to eliminate context bias and improve the multi-modal representation. We design a new question-only branch based on counterfactual generation to distill and eliminate keyword bias.
  arXiv  Detail & Related papers  (2024-10-14T06:09:16Z)
• GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Models [75.04426753720553]
  We propose a framework to identify, quantify, and explain biases in an open set setting. This pipeline leverages a Large Language Model (LLM) to propose biases starting from a set of captions. We show two variations of this framework: OpenBias and GradBias.
  arXiv  Detail & Related papers  (2024-08-29T16:51:07Z)
• Projective Methods for Mitigating Gender Bias in Pre-trained Language Models [10.418595661963062]
  Projective methods are fast to implement, use a small number of saved parameters, and make no updates to the existing model parameters. We find that projective methods can be effective at both intrinsic bias and downstream bias mitigation, but that the two outcomes are not necessarily correlated.
  arXiv  Detail & Related papers  (2024-03-27T17:49:31Z)
• Is There a One-Model-Fits-All Approach to Information Extraction? Revisiting Task Definition Biases [62.806300074459116]
  Definition bias is a negative phenomenon that can mislead models. We identify two types of definition bias in IE: bias among information extraction datasets and bias between information extraction datasets and instruction tuning datasets. We propose a multi-stage framework consisting of definition bias measurement, bias-aware fine-tuning, and task-specific bias mitigation.
  arXiv  Detail & Related papers  (2024-03-25T03:19:20Z)
• Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction [56.17020601803071]
  Recent research shows that pre-trained language models (PLMs) suffer from "prompt bias" in factual knowledge extraction. This paper aims to improve the reliability of existing benchmarks by thoroughly investigating and mitigating prompt bias.
  arXiv  Detail & Related papers  (2024-03-15T02:04:35Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• Target-Aware Contextual Political Bias Detection in News [22.396285428304083]
  Sentence-level political bias detection in news is a challenging task that requires an understanding of bias in consideration of the context. Previous work in media bias detection has proposed augmentation techniques to exploit this fact. We propose techniques to more carefully search for context using a bias-sensitive, target-aware approach for data augmentation.
  arXiv  Detail & Related papers  (2023-10-02T12:25:05Z)
• Discovering and Mitigating Visual Biases through Keyword Explanation [66.71792624377069]
  We propose the Bias-to-Text (B2T) framework, which interprets visual biases as keywords. B2T can identify known biases, such as gender bias in CelebA, background bias in Waterbirds, and distribution shifts in ImageNet-R/C. B2T uncovers novel biases in larger datasets, such as Dollar Street and ImageNet.
  arXiv  Detail & Related papers  (2023-01-26T13:58:46Z)
• Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By Experts [24.51774048437496]
  This paper presents BABE, a robust and diverse data set for media bias research. It consists of 3,700 sentences balanced among topics and outlets, containing media bias labels on the word and sentence level. Based on our data, we also introduce a way to detect bias-inducing sentences in news articles automatically.
  arXiv  Detail & Related papers  (2022-09-29T05:32:55Z)
• The SAME score: Improved cosine based bias score for word embeddings [49.75878234192369]
  We introduce SAME, a novel bias score for semantic bias in embeddings. We show that SAME is capable of measuring semantic bias and identify potential causes for social bias in downstream tasks.
  arXiv  Detail & Related papers  (2022-03-28T09:28:13Z)
• "Thy algorithm shalt not bear false witness": An Evaluation of Multiclass Debiasing Methods on Word Embeddings [3.0204693431381515]
  The paper investigates the state-of-the-art multiclass debiasing techniques: Hard debiasing, SoftWEAT debiasing and Conceptor debiasing. It evaluates their performance when removing religious bias on a common basis by quantifying bias removal via the Word Embedding Association Test (WEAT), Mean Average Cosine Similarity (MAC) and the Relative Negative Sentiment Bias (RNSB)
  arXiv  Detail & Related papers  (2020-10-30T12:49:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.