Enhancing Interpretability using Human Similarity Judgements to Prune
Word Embeddings
- URL: http://arxiv.org/abs/2310.10262v1
- Date: Mon, 16 Oct 2023 10:38:49 GMT
- Title: Enhancing Interpretability using Human Similarity Judgements to Prune
Word Embeddings
- Authors: Natalia Flechas Manrique, Wanqian Bao, Aurelie Herbelot, Uri Hasson
- Abstract summary: Interpretability methods in NLP aim to provide insights into the semantics underlying specific system architectures.
We present a supervised-learning method that identifies a subset of model features that strongly improve prediction of human similarity judgments.
We show this method keeps only 20-40% of the original embeddings, for 8 independent semantic domains.
We then present two approaches for interpreting the semantics of the retained features.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Interpretability methods in NLP aim to provide insights into the semantics
underlying specific system architectures. Focusing on word embeddings, we
present a supervised-learning method that, for a given domain (e.g., sports,
professions), identifies a subset of model features that strongly improve
prediction of human similarity judgments. We show this method keeps only 20-40%
of the original embeddings, for 8 independent semantic domains, and that it
retains different feature sets across domains. We then present two approaches
for interpreting the semantics of the retained features. The first obtains the
scores of the domain words (co-hyponyms) on the first principal component of
the retained embeddings, and extracts terms whose co-occurrence with the
co-hyponyms tracks these scores' profile. This analysis reveals that humans
differentiate e.g. sports based on how gender-inclusive and international they
are. The second approach uses the retained sets as variables in a probing task
that predicts values along 65 semantically annotated dimensions for a dataset
of 535 words. The features retained for professions are best at predicting
cognitive, emotional and social dimensions, whereas features retained for
fruits or vegetables best predict the gustation (taste) dimension. We discuss
implications for alignment between AI systems and human knowledge.
Related papers
- Identifying and interpreting non-aligned human conceptual
representations using language modeling [0.0]
We show that congenital blindness induces conceptual reorganization in both a-modal and sensory-related verbal domains.
We find that blind individuals more strongly associate social and cognitive meanings to verbs related to motion.
For some verbs, representations of blind and sighted are highly similar.
arXiv Detail & Related papers (2024-03-10T13:02:27Z) - Domain Embeddings for Generating Complex Descriptions of Concepts in
Italian Language [65.268245109828]
We propose a Distributional Semantic resource enriched with linguistic and lexical information extracted from electronic dictionaries.
The resource comprises 21 domain-specific matrices, one comprehensive matrix, and a Graphical User Interface.
Our model facilitates the generation of reasoned semantic descriptions of concepts by selecting matrices directly associated with concrete conceptual knowledge.
arXiv Detail & Related papers (2024-02-26T15:04:35Z) - Unsupervised Semantic Variation Prediction using the Distribution of
Sibling Embeddings [17.803726860514193]
Detection of semantic variation of words is an important task for various NLP applications.
We argue that mean representations alone cannot accurately capture such semantic variations.
We propose a method that uses the entire cohort of the contextualised embeddings of the target word.
arXiv Detail & Related papers (2023-05-15T13:58:21Z) - Semantic Human Parsing via Scalable Semantic Transfer over Multiple
Label Domains [25.083197183341007]
This paper presents a novel training paradigm to train a powerful human parsing network.
Two common application scenarios are addressed, termed universal parsing and dedicated parsing.
Experimental results demonstrate SST can effectively achieve promising universal human parsing performance.
arXiv Detail & Related papers (2023-04-09T02:44:29Z) - Textual Entailment Recognition with Semantic Features from Empirical
Text Representation [60.31047947815282]
A text entails a hypothesis if and only if the true value of the hypothesis follows the text.
In this paper, we propose a novel approach to identifying the textual entailment relationship between text and hypothesis.
We employ an element-wise Manhattan distance vector-based feature that can identify the semantic entailment relationship between the text-hypothesis pair.
arXiv Detail & Related papers (2022-10-18T10:03:51Z) - A Latent-Variable Model for Intrinsic Probing [93.62808331764072]
We propose a novel latent-variable formulation for constructing intrinsic probes.
We find empirical evidence that pre-trained representations develop a cross-lingually entangled notion of morphosyntax.
arXiv Detail & Related papers (2022-01-20T15:01:12Z) - Fake it Till You Make it: Self-Supervised Semantic Shifts for
Monolingual Word Embedding Tasks [58.87961226278285]
We propose a self-supervised approach to model lexical semantic change.
We show that our method can be used for the detection of semantic change with any alignment method.
We illustrate the utility of our techniques using experimental results on three different datasets.
arXiv Detail & Related papers (2021-01-30T18:59:43Z) - Infusing Finetuning with Semantic Dependencies [62.37697048781823]
We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models.
We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
arXiv Detail & Related papers (2020-12-10T01:27:24Z) - Weakly-Supervised Aspect-Based Sentiment Analysis via Joint
Aspect-Sentiment Topic Embedding [71.2260967797055]
We propose a weakly-supervised approach for aspect-based sentiment analysis.
We learn sentiment, aspect> joint topic embeddings in the word embedding space.
We then use neural models to generalize the word-level discriminative information.
arXiv Detail & Related papers (2020-10-13T21:33:24Z) - On the Learnability of Concepts: With Applications to Comparing Word
Embedding Algorithms [0.0]
We introduce the notion of "concept" as a list of words that have shared semantic content.
We first use this notion to measure the learnability of concepts on pretrained word embeddings.
We then develop a statistical analysis of concept learnability, based on hypothesis testing and ROC curves, in order to compare the relative merits of various embedding algorithms.
arXiv Detail & Related papers (2020-06-17T14:25:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.