Domain-Specific Word Embeddings with Structure Prediction

• URL: http://arxiv.org/abs/2210.04962v1
• Date: Thu, 6 Oct 2022 12:45:48 GMT
• Title: Domain-Specific Word Embeddings with Structure Prediction
• Authors: Stephanie Brandl and David Lassner and Anne Baillot and Shinichi Nakajima
• Abstract summary: We present an empirical evaluation on New York Times articles and two English Wikipedia datasets with articles on science and philosophy. Our method, called Word2Vec with Structure Prediction (W2VPred), provides better performance than baselines in terms of the general analogy tests. As a use case in the field of Digital Humanities we demonstrate how to raise novel research questions for high literature from the German Text Archive.
• Score: 3.057136788672694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Complementary to finding good general word embeddings, an important question for representation learning is to find dynamic word embeddings, e.g., across time or domain. Current methods do not offer a way to use or predict information on structure between sub-corpora, time or domain and dynamic embeddings can only be compared after post-alignment. We propose novel word embedding methods that provide general word representations for the whole corpus, domain-specific representations for each sub-corpus, sub-corpus structure, and embedding alignment simultaneously. We present an empirical evaluation on New York Times articles and two English Wikipedia datasets with articles on science and philosophy. Our method, called Word2Vec with Structure Prediction (W2VPred), provides better performance than baselines in terms of the general analogy tests, domain-specific analogy tests, and multiple specific word embedding evaluations as well as structure prediction performance when no structure is given a priori. As a use case in the field of Digital Humanities we demonstrate how to raise novel research questions for high literature from the German Text Archive.

Related papers

• Are we describing the same sound? An analysis of word embedding spaces of expressive piano performance [4.867952721052875]
  We investigate the uncertainty for the domain of characterizations of expressive piano performance. We test five embedding models and their similarity structure for correspondence with the ground truth. The quality of embedding models shows great variability with respect to this task.
  arXiv  Detail & Related papers  (2023-12-31T12:20:03Z)
• Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval [21.262531222066208]
  In this paper, we devise four pre-training objectives tailored for information retrieval tasks based on the structured knowledge of Wikipedia. Compared to existing pre-training methods, our approach can better capture the semantic knowledge in the training corpus. Experimental results in biomedical and legal domains demonstrate that our approach achieves better performance in vertical domains.
  arXiv  Detail & Related papers  (2023-12-17T09:31:47Z)
• Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding [143.5927158318524]
  Temporal grounding is the task of locating a specific segment from an untrimmed video according to a query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We argue that the inherent structured semantics inside the videos and language is the crucial factor to achieve compositional generalization.
  arXiv  Detail & Related papers  (2023-01-22T08:02:23Z)
• PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition [63.51569687229681]
  We argue for the need to recognize the textual entailment relation of each proposition in a sentence individually. We propose PropSegmEnt, a corpus of over 45K propositions annotated by expert human raters. Our dataset structure resembles the tasks of (1) segmenting sentences within a document to the set of propositions, and (2) classifying the entailment relation of each proposition with respect to a different yet topically-aligned document.
  arXiv  Detail & Related papers  (2022-12-21T04:03:33Z)
• Textual Entailment Recognition with Semantic Features from Empirical Text Representation [60.31047947815282]
  A text entails a hypothesis if and only if the true value of the hypothesis follows the text. In this paper, we propose a novel approach to identifying the textual entailment relationship between text and hypothesis. We employ an element-wise Manhattan distance vector-based feature that can identify the semantic entailment relationship between the text-hypothesis pair.
  arXiv  Detail & Related papers  (2022-10-18T10:03:51Z)
• Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning [92.07643510310766]
  Temporal grounding in videos aims to localize one target video segment that semantically corresponds to a given query sentence. We introduce a new Compositional Temporal Grounding task and construct two new dataset splits. We empirically find that they fail to generalize to queries with novel combinations of seen words. We propose a variational cross-graph reasoning framework that explicitly decomposes video and language into multiple structured hierarchies.
  arXiv  Detail & Related papers  (2022-03-24T12:55:23Z)
• More Romanian word embeddings from the RETEROM project [0.0]
  "word embeddings" are automatically learned vector representations of words. We plan to develop an openaccess large library of ready-to-use word embeddings sets.
  arXiv  Detail & Related papers  (2021-11-21T06:05:12Z)
• Clustering and Network Analysis for the Embedding Spaces of Sentences and Sub-Sentences [69.3939291118954]
  This paper reports research on a set of comprehensive clustering and network analyses targeting sentence and sub-sentence embedding spaces. Results show that one method generates the most clusterable embeddings. In general, the embeddings of span sub-sentences have better clustering properties than the original sentences.
  arXiv  Detail & Related papers  (2021-10-02T00:47:35Z)
• A Self-supervised Representation Learning of Sentence Structure for Authorship Attribution [3.5991811164452923]
  We propose a self-supervised framework for learning structural representations of sentences. We evaluate the learned structural representations of sentences using different probing tasks, and subsequently utilize them in the authorship attribution task.
  arXiv  Detail & Related papers  (2020-10-14T02:57:10Z)
• A Comparative Study on Structural and Semantic Properties of Sentence Embeddings [77.34726150561087]
  We propose a set of experiments using a widely-used large-scale data set for relation extraction. We show that different embedding spaces have different degrees of strength for the structural and semantic properties. These results provide useful information for developing embedding-based relation extraction methods.
  arXiv  Detail & Related papers  (2020-09-23T15:45:32Z)
• Compass-aligned Distributional Embeddings for Studying Semantic Differences across Corpora [14.993021283916008]
  We present a framework to support cross-corpora language studies with word embeddings. CADE is the core component of our framework and solves the key problem of aligning the embeddings generated from different corpora. The results of our experiments suggest that CADE achieves state-of-the-art or superior performance on tasks where several competing approaches are available.
  arXiv  Detail & Related papers  (2020-04-13T15:46:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.