Towards a Flexible Embedding Learning Framework

• URL: http://arxiv.org/abs/2009.10989v1
• Date: Wed, 23 Sep 2020 08:00:56 GMT
• Title: Towards a Flexible Embedding Learning Framework
• Authors: Chin-Chia Michael Yeh, Dhruv Gelda, Zhongfang Zhuang, Yan Zheng, Liang Gou, Wei Zhang
• Abstract summary: We propose an embedding learning framework that is flexible in terms of the relationships that can be embedded into the learned representations. A sampling mechanism is carefully designed to establish a direct connection between the input and the information captured by the output embeddings. Our empirical results demonstrate that the proposed framework, in conjunction with a set of relevant entity-relation-matrices, outperforms the existing state-of-the-art approaches in various data mining tasks.
• Score: 15.604564543883122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Representation learning is a fundamental building block for analyzing entities in a database. While the existing embedding learning methods are effective in various data mining problems, their applicability is often limited because these methods have pre-determined assumptions on the type of semantics captured by the learned embeddings, and the assumptions may not well align with specific downstream tasks. In this work, we propose an embedding learning framework that 1) uses an input format that is agnostic to input data type, 2) is flexible in terms of the relationships that can be embedded into the learned representations, and 3) provides an intuitive pathway to incorporate domain knowledge into the embedding learning process. Our proposed framework utilizes a set of entity-relation-matrices as the input, which quantifies the affinities among different entities in the database. Moreover, a sampling mechanism is carefully designed to establish a direct connection between the input and the information captured by the output embeddings. To complete the representation learning toolbox, we also outline a simple yet effective post-processing technique to properly visualize the learned embeddings. Our empirical results demonstrate that the proposed framework, in conjunction with a set of relevant entity-relation-matrices, outperforms the existing state-of-the-art approaches in various data mining tasks.

Related papers

• Informed Meta-Learning [55.2480439325792]
  Meta-learning and informed ML stand out as two approaches for incorporating prior knowledge into ML pipelines. We formalise a hybrid paradigm, informed meta-learning, facilitating the incorporation of priors from unstructured knowledge representations. We demonstrate the potential benefits of informed meta-learning in improving data efficiency, robustness to observational noise and task distribution shifts.
  arXiv  Detail & Related papers  (2024-02-25T15:08:37Z)
• Learning Representations without Compositional Assumptions [79.12273403390311]
  We propose a data-driven approach that learns feature set dependencies by representing feature sets as graph nodes and their relationships as learnable edges. We also introduce LEGATO, a novel hierarchical graph autoencoder that learns a smaller, latent graph to aggregate information from multiple views dynamically.
  arXiv  Detail & Related papers  (2023-05-31T10:36:10Z)
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [57.854498238624366]
  We propose a retrieval-augmented approach, which retrieves schema-aware Reference As Prompt (RAP) for data-efficient knowledge graph construction. RAP can dynamically leverage schema and knowledge inherited from human-annotated and weak-supervised data as a prompt for each sample.
  arXiv  Detail & Related papers  (2022-10-19T16:40:28Z)
• Representing Knowledge by Spans: A Knowledge-Enhanced Model for Information Extraction [7.077412533545456]
  We propose a new pre-trained model that learns representations of both entities and relationships simultaneously. By encoding spans efficiently with span modules, our model can represent both entities and their relationships but requires fewer parameters than existing models.
  arXiv  Detail & Related papers  (2022-08-20T07:32:25Z)
• Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
  We tackle the problem of learning visual representations from unlabeled scene-centric data. We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
  arXiv  Detail & Related papers  (2022-05-30T17:50:59Z)
• Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks [0.0]
  We propose a versatile, unified framework called HMill' for sample representation, model definition and training. We show an extension of the universal approximation theorem to the set of all functions realized by models implemented in the framework. We solve three different problems from the cybersecurity domain using the framework.
  arXiv  Detail & Related papers  (2021-04-19T21:32:44Z)
• Multifaceted Context Representation using Dual Attention for Ontology Alignment [6.445605125467574]
  Ontology alignment is an important research problem that finds application in various fields such as data integration, data transfer, data preparation etc. We propose VeeAlign, a Deep Learning based model that uses a dual-attention mechanism to compute the contextualized representation of a concept in order to learn alignments. We validate our approach on various datasets from different domains and in multilingual settings, and show its superior performance over SOTA methods.
  arXiv  Detail & Related papers  (2020-10-16T18:28:38Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• Propositionalization and Embeddings: Two Sides of the Same Coin [0.0]
  This paper outlines some of the modern data processing techniques used in relational learning. It focuses on the propositionalization and embedding data transformation approaches. We present two efficient implementations of the unifying methodology.
  arXiv  Detail & Related papers  (2020-06-08T08:33:21Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.