The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings

• URL: http://arxiv.org/abs/2001.09876v2
• Date: Tue, 28 Jan 2020 13:40:53 GMT
• Title: The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings
• Authors: Binny Mathew, Sandipan Sikdar, Florian Lemmerich and Markus Strohmaier
• Abstract summary: We introduce POLAR - a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials. We demonstrate the effectiveness of our framework by deploying it to various downstream tasks. We also show that the interpretable dimensions selected by our framework align with human judgement.
• Score: 6.894744675083238
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce POLAR - a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials. Semantic differentials are a psychometric construct for measuring the semantics of a word by analysing its position on a scale between two polar opposites (e.g., cold -- hot, soft -- hard). The core idea of our approach is to transform existing, pre-trained word embeddings via semantic differentials to a new "polar" space with interpretable dimensions defined by such polar opposites. Our framework also allows for selecting the most discriminative dimensions from a set of polar dimensions provided by an oracle, i.e., an external source. We demonstrate the effectiveness of our framework by deploying it to various downstream tasks, in which our interpretable word embeddings achieve a performance that is comparable to the original word embeddings. We also show that the interpretable dimensions selected by our framework align with human judgement. Together, these results demonstrate that interpretability can be added to word embeddings without compromising performance. Our work is relevant for researchers and engineers interested in interpreting pre-trained word embeddings.

Related papers

• Enhancing Interpretability using Human Similarity Judgements to Prune Word Embeddings [0.0]
  Interpretability methods in NLP aim to provide insights into the semantics underlying specific system architectures. We present a supervised-learning method that identifies a subset of model features that strongly improve prediction of human similarity judgments. We show this method keeps only 20-40% of the original embeddings, for 8 independent semantic domains. We then present two approaches for interpreting the semantics of the retained features.
  arXiv  Detail & Related papers  (2023-10-16T10:38:49Z)
• Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations [80.45474362071236]
  It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space. We propose InterSent, an end-to-end framework for learning interpretable sentence embeddings.
  arXiv  Detail & Related papers  (2023-05-24T00:44:49Z)
• Unsupervised Interpretable Basis Extraction for Concept-Based Visual Explanations [53.973055975918655]
  We show that, intermediate layer representations become more interpretable when transformed to the bases extracted with our method. We compare the bases extracted with our method with the bases derived with a supervised approach and find that, in one aspect, the proposed unsupervised approach has a strength that constitutes a limitation of the supervised one and give potential directions for future research.
  arXiv  Detail & Related papers  (2023-03-19T00:37:19Z)
• SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings [4.479834103607384]
  Adding interpretability to word embeddings represents an area of active research in text representation. We present SensePOLAR, an extension of the original POLAR framework that enables word-sense aware interpretability for pre-trained contextual word embeddings.
  arXiv  Detail & Related papers  (2023-01-11T20:25:53Z)
• Relational Sentence Embedding for Flexible Semantic Matching [86.21393054423355]
  We present Sentence Embedding (RSE), a new paradigm to discover further the potential of sentence embeddings. RSE is effective and flexible in modeling sentence relations and outperforms a series of state-of-the-art embedding methods.
  arXiv  Detail & Related papers  (2022-12-17T05:25:17Z)
• Robust Unsupervised Cross-Lingual Word Embedding using Domain Flow Interpolation [48.32604585839687]
  Previous adversarial approaches have shown promising results in inducing cross-lingual word embedding without parallel data. We propose to make use of a sequence of intermediate spaces for smooth bridging.
  arXiv  Detail & Related papers  (2022-10-07T04:37:47Z)
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation [59.01297461453444]
  We propose a hierarchical contrastive learning mechanism, which can unify hybrid granularities semantic meaning in the input text. Experiments demonstrate that our model outperforms competitive baselines on paraphrasing, dialogue generation, and storytelling tasks.
  arXiv  Detail & Related papers  (2022-05-26T13:26:03Z)
• Human-in-the-Loop Refinement of Word Embeddings [0.0]
  We propose a system that incorporates an adaptation of word embedding post-processing, which we call "interactive refitting" Our approach allows a human to identify and address potential quality issues with word embeddings interactively. It also allows for better insight into what effect word embeddings, and refinements to word embeddings, have on machine learning pipelines.
  arXiv  Detail & Related papers  (2021-10-06T16:10:32Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• Compass-aligned Distributional Embeddings for Studying Semantic Differences across Corpora [14.993021283916008]
  We present a framework to support cross-corpora language studies with word embeddings. CADE is the core component of our framework and solves the key problem of aligning the embeddings generated from different corpora. The results of our experiments suggest that CADE achieves state-of-the-art or superior performance on tasks where several competing approaches are available.
  arXiv  Detail & Related papers  (2020-04-13T15:46:47Z)
• Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation [25.060917870666803]
  We introduce a siamese auto-encoder structure with an adapted gradient reversal layer. Our structure enables the separation of the semantic latent information and gender latent information of given word into the disjoint latent dimensions.
  arXiv  Detail & Related papers  (2020-04-07T05:16:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.