Discovering Differences in the Representation of People using Contextualized Semantic Axes

• URL: http://arxiv.org/abs/2210.12170v1
• Date: Fri, 21 Oct 2022 18:02:19 GMT
• Title: Discovering Differences in the Representation of People using Contextualized Semantic Axes
• Authors: Li Lucy, Divya Tadimeti, David Bamman
• Abstract summary: We use contextualized semantic axes to characterize differences among instances of the same word type. We show that references to women and the contexts around them have become more detestable over time.
• Score: 5.972927416266617
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A common paradigm for identifying semantic differences across social and temporal contexts is the use of static word embeddings and their distances. In particular, past work has compared embeddings against "semantic axes" that represent two opposing concepts. We extend this paradigm to BERT embeddings, and construct contextualized axes that mitigate the pitfall where antonyms have neighboring representations. We validate and demonstrate these axes on two people-centric datasets: occupations from Wikipedia, and multi-platform discussions in extremist, men's communities over fourteen years. In both studies, contextualized semantic axes can characterize differences among instances of the same word type. In the latter study, we show that references to women and the contexts around them have become more detestable over time.

Related papers

• Conjuring Semantic Similarity [59.18714889874088]
  The semantic similarity between two textual expressions measures the distance between their latent'meaning' We propose a novel approach whereby the semantic similarity among textual expressions is based not on other expressions they can be rephrased as, but rather based on the imagery they evoke. Our method contributes a novel perspective on semantic similarity that not only aligns with human-annotated scores, but also opens up new avenues for the evaluation of text-conditioned generative models.
  arXiv  Detail & Related papers  (2024-10-21T18:51:34Z)
• Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning [71.14084801851381]
  Change captioning aims to succinctly describe the semantic change between a pair of similar images. Most existing methods directly capture the difference between them, which risk obtaining error-prone difference features. We propose a distractors-immune representation learning network that correlates the corresponding channels of two image representations.
  arXiv  Detail & Related papers  (2024-07-16T13:00:33Z)
• A Tale of Two Laws of Semantic Change: Predicting Synonym Changes with Distributional Semantic Models [1.856334276134661]
  There are two competing, apparently opposite hypotheses in the historical linguistic literature regarding how synonymous words evolve. We take a first step toward detecting whether Law of Differentiation (LD) or Law of Parallel Change (LPC) operates for given word pairs. We then propose various computational approaches to the problem using Distributional Semantic Models and grounded in recent literature on Lexical Semantic Change detection.
  arXiv  Detail & Related papers  (2023-05-30T15:50:29Z)
• Dialectograms: Machine Learning Differences between Discursive Communities [0.0]
  We take a step towards leveraging the richness of the full embedding space by using word embeddings to map out how words are used differently. We provide a new measure of the degree to which words are used differently that overcomes the tendency for existing measures to pick out low frequent or polysemous words.
  arXiv  Detail & Related papers  (2023-02-11T11:32:08Z)
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation [59.01297461453444]
  We propose a hierarchical contrastive learning mechanism, which can unify hybrid granularities semantic meaning in the input text. Experiments demonstrate that our model outperforms competitive baselines on paraphrasing, dialogue generation, and storytelling tasks.
  arXiv  Detail & Related papers  (2022-05-26T13:26:03Z)
• Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning [92.07643510310766]
  Temporal grounding in videos aims to localize one target video segment that semantically corresponds to a given query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We empirically find that they fail to generalize to queries with novel combinations of seen words. We propose a variational cross-graph reasoning framework that explicitly decomposes video and language into multiple structured hierarchies.
  arXiv  Detail & Related papers  (2022-03-24T12:55:23Z)
• Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy [0.0]
  We assess the ability of both static and contextualized models to adequately represent different lexical-semantic relations. Experiments are performed in Galician, Portuguese, English, and Spanish.
  arXiv  Detail & Related papers  (2021-06-25T10:54:23Z)
• Frequency-based Distortions in Contextualized Word Embeddings [29.88883761339757]
  This work explores the geometric characteristics of contextualized word embeddings with two novel tools. Words of high and low frequency differ significantly with respect to their representational geometry. BERT-Base has more trouble differentiating between South American and African countries than North American and European ones.
  arXiv  Detail & Related papers  (2021-04-17T06:35:48Z)
• Fake it Till You Make it: Self-Supervised Semantic Shifts for Monolingual Word Embedding Tasks [58.87961226278285]
  We propose a self-supervised approach to model lexical semantic change. We show that our method can be used for the detection of semantic change with any alignment method. We illustrate the utility of our techniques using experimental results on three different datasets.
  arXiv  Detail & Related papers  (2021-01-30T18:59:43Z)
• Lexical semantic change for Ancient Greek and Latin [61.69697586178796]
  Associating a word's correct meaning in its historical context is a central challenge in diachronic research. We build on a recent computational approach to semantic change based on a dynamic Bayesian mixture model. We provide a systematic comparison of dynamic Bayesian mixture models for semantic change with state-of-the-art embedding-based models.
  arXiv  Detail & Related papers  (2021-01-22T12:04:08Z)
• Cultural Cartography with Word Embeddings [0.0]
  We show how word embeddings are commensurate with prevailing theories of meaning in sociology. First, one can hold terms constant and measure how the embedding space moves around them. Second, one can also hold the embedding space constant and see how documents or authors move relative to it.
  arXiv  Detail & Related papers  (2020-07-09T01:58:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.