Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology

• URL: http://arxiv.org/abs/2212.07547v1
• Date: Wed, 14 Dec 2022 23:31:14 GMT
• Title: Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology
• Authors: Valentin Hofmann, Janet B. Pierrehumbert, Hinrich Sch\"utze
• Abstract summary: We propose a fully unsupervised method to detect bias in contextualized embeddings. We show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.
• Score: 20.81930455526026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace capturing this information. As a concrete example, we focus on the phenomenon of ideological bias: we introduce the concept of an ideological subspace, show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.

Related papers

• Encoding Hierarchical Schema via Concept Flow for Multifaceted Ideology Detection [26.702058189138462]
  Multifaceted ideology detection (MID) aims to detect the ideological leanings of texts towards multiple facets. We develop a novel concept semantics-enhanced framework for the MID task. Our approach achieves state-of-the-art performance in MID, including in the cross-topic scenario.
  arXiv  Detail & Related papers  (2024-05-29T10:37:28Z)
• Understanding Distributed Representations of Concepts in Deep Neural Networks without Supervision [25.449397570387802]
  We propose an unsupervised method for discovering distributed representations of concepts by selecting a principal subset of neurons. Our empirical findings demonstrate that instances with similar neuron activation states tend to share coherent concepts. It can be utilized to identify unlabeled subclasses within data and to detect the causes of misclassifications.
  arXiv  Detail & Related papers  (2023-12-28T07:33:51Z)
• Unsupervised Interpretable Basis Extraction for Concept-Based Visual Explanations [53.973055975918655]
  We show that, intermediate layer representations become more interpretable when transformed to the bases extracted with our method. We compare the bases extracted with our method with the bases derived with a supervised approach and find that, in one aspect, the proposed unsupervised approach has a strength that constitutes a limitation of the supervised one and give potential directions for future research.
  arXiv  Detail & Related papers  (2023-03-19T00:37:19Z)
• Bayesian Learning for Neural Networks: an algorithmic survey [95.42181254494287]
  This self-contained survey engages and introduces readers to the principles and algorithms of Bayesian Learning for Neural Networks. It provides an introduction to the topic from an accessible, practical-algorithmic perspective.
  arXiv  Detail & Related papers  (2022-11-21T21:36:58Z)
• Imitation Learning-based Implicit Semantic-aware Communication Networks: Multi-layer Representation and Collaborative Reasoning [68.63380306259742]
  Despite its promising potential, semantic communications and semantic-aware networking are still at their infancy. We propose a novel reasoning-based implicit semantic-aware communication network architecture that allows multiple tiers of CDC and edge servers to collaborate. We introduce a new multi-layer representation of semantic information taking into consideration both the hierarchical structure of implicit semantics as well as the personalized inference preference of individual users.
  arXiv  Detail & Related papers  (2022-10-28T13:26:08Z)
• Latent Topology Induction for Understanding Contextualized Representations [84.7918739062235]
  We study the representation space of contextualized embeddings and gain insight into the hidden topology of large language models. We show there exists a network of latent states that summarize linguistic properties of contextualized representations.
  arXiv  Detail & Related papers  (2022-06-03T11:22:48Z)
• KCD: Knowledge Walks and Textual Cues Enhanced Political Perspective Detection in News Media [28.813287482918344]
  We propose KCD, a political perspective detection approach to enable multi-hop knowledge reasoning. Specifically, we generate random walks on external knowledge graphs and infuse them with news text representations. We then construct a heterogeneous information network to jointly model news content as well as semantic, syntactic and entity cues in news articles.
  arXiv  Detail & Related papers  (2022-04-08T13:06:09Z)
• TopicNet: Semantic Graph-Guided Topic Discovery [51.71374479354178]
  Existing deep hierarchical topic models are able to extract semantically meaningful topics from a text corpus in an unsupervised manner. We introduce TopicNet as a deep hierarchical topic model that can inject prior structural knowledge as an inductive bias to influence learning.
  arXiv  Detail & Related papers  (2021-10-27T09:07:14Z)
• Learning Ideological Embeddings from Information Cascades [11.898833102736255]
  We propose a model to learn the ideological leaning of each user in a multidimensional ideological space. Our model is able to learn the political stance of the social media users in a multidimensional ideological space.
  arXiv  Detail & Related papers  (2021-09-28T09:58:02Z)
• A Comprehensive Survey on Community Detection with Deep Learning [93.40332347374712]
  A community reveals the features and connections of its members that are different from those in other communities in a network. This survey devises and proposes a new taxonomy covering different categories of the state-of-the-art methods. The main category, i.e., deep neural networks, is further divided into convolutional networks, graph attention networks, generative adversarial networks and autoencoders.
  arXiv  Detail & Related papers  (2021-05-26T14:37:07Z)
• Modeling Ideological Agenda Setting and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity [13.535770763481905]
  We introduce a minimally supervised method that directly leverages the network structure of online discussion forums, specifically Reddit, to detect polarized concepts. We model polarization along the dimensions of agenda setting and framing, drawing upon insights from moral psychology. We also create a new dataset of political discourse spanning 12 years and covering more than 600 online groups with different ideologies.
  arXiv  Detail & Related papers  (2021-04-18T11:48:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.