Grounding and Distinguishing Conceptual Vocabulary Through Similarity Learning in Embodied Simulations

• URL: http://arxiv.org/abs/2305.13668v1
• Date: Tue, 23 May 2023 04:22:00 GMT
• Title: Grounding and Distinguishing Conceptual Vocabulary Through Similarity Learning in Embodied Simulations
• Authors: Sadaf Ghaffari and Nikhil Krishnaswamy
• Abstract summary: We present a novel method for using agent experiences gathered through an embodied simulation to ground contextualized word vectors to object representations. We use similarity learning to make comparisons between different object types based on their properties when interacted with, and to extract common features pertaining to the objects' behavior.
• Score: 4.507860128918788
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel method for using agent experiences gathered through an embodied simulation to ground contextualized word vectors to object representations. We use similarity learning to make comparisons between different object types based on their properties when interacted with, and to extract common features pertaining to the objects' behavior. We then use an affine transformation to calculate a projection matrix that transforms contextualized word vectors from different transformer-based language models into this learned space, and evaluate whether new test instances of transformed token vectors identify the correct concept in the object embedding space. Our results expose properties of the embedding spaces of four different transformer models and show that grounding object token vectors is usually more helpful to grounding verb and attribute token vectors than the reverse, which reflects earlier conclusions in the analogical reasoning and psycholinguistic literature.

Related papers

• Local Topology Measures of Contextual Language Model Latent Spaces With Applications to Dialogue Term Extraction [4.887047578768969]
  We introduce complexity measures of the local topology of the latent space of a contextual language model. Our work continues a line of research that explores the manifold hypothesis for word embeddings.
  arXiv  Detail & Related papers  (2024-08-07T11:44:32Z)
• An Intrinsic Vector Heat Network [64.55434397799728]
  This paper introduces a novel neural network architecture for learning tangent vector fields embedded in 3D. We introduce a trainable vector heat diffusion module to spatially propagate vector-valued feature data across the surface. We also demonstrate the effectiveness of our method on the useful industrial application of quadrilateral mesh generation.
  arXiv  Detail & Related papers  (2024-06-14T00:40:31Z)
• Observable Propagation: Uncovering Feature Vectors in Transformers [21.2431937128876]
  We introduce "observable propagation" (in short: ObProp) for finding linear features used by transformer language models in computing a given task. Our paradigm centers on the concept of "observables", linear functionals corresponding to given tasks. We use ObProp to perform extensive qualitative investigations into several tasks, including gendered occupational bias, political party prediction, and programming language detection.
  arXiv  Detail & Related papers  (2023-12-26T19:00:56Z)
• Backpack Language Models [108.65930795825416]
  We present Backpacks, a new neural architecture that marries strong modeling performance with an interface for interpretability and control. We find that, after training, sense vectors specialize, each encoding a different aspect of a word. We present simple algorithms that intervene on sense vectors to perform controllable text generation and debiasing.
  arXiv  Detail & Related papers  (2023-05-26T09:26:23Z)
• Understanding Synonymous Referring Expressions via Contrastive Features [105.36814858748285]
  We develop an end-to-end trainable framework to learn contrastive features on the image and object instance levels. We conduct extensive experiments to evaluate the proposed algorithm on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-20T17:56:24Z)
• Prototypical Representation Learning for Relation Extraction [56.501332067073065]
  This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data. We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
  arXiv  Detail & Related papers  (2021-03-22T08:11:43Z)
• Image Captioning with Visual Object Representations Grounded in the Textual Modality [14.797241131469486]
  We explore the possibilities of a shared embedding space between textual and visual modality. We propose an approach opposite to the current trend, grounding of the representations in the word embedding space of the captioning system.
  arXiv  Detail & Related papers  (2020-10-19T12:21:38Z)
• Unsupervised Distillation of Syntactic Information from Contextualized Word Representations [62.230491683411536]
  We tackle the task of unsupervised disentanglement between semantics and structure in neural language representations. To this end, we automatically generate groups of sentences which are structurally similar but semantically different. We demonstrate that our transformation clusters vectors in space by structural properties, rather than by lexical semantics.
  arXiv  Detail & Related papers  (2020-10-11T15:13:18Z)
• Learning Universal Representations from Word to Sentence [89.82415322763475]
  This work introduces and explores the universal representation learning, i.e., embeddings of different levels of linguistic unit in a uniform vector space. We present our approach of constructing analogy datasets in terms of words, phrases and sentences. We empirically verify that well pre-trained Transformer models incorporated with appropriate training settings may effectively yield universal representation.
  arXiv  Detail & Related papers  (2020-09-10T03:53:18Z)
• Principal Word Vectors [5.64434321651888]
  We generalize principal component analysis for embedding words into a vector space. We show that the spread and the discriminability of the principal word vectors are higher than that of other word embedding methods.
  arXiv  Detail & Related papers  (2020-07-09T08:29:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.