Multi-Partition Embedding Interaction with Block Term Format for Knowledge Graph Completion

• URL: http://arxiv.org/abs/2006.16365v2
• Date: Sat, 1 Oct 2022 20:26:22 GMT
• Title: Multi-Partition Embedding Interaction with Block Term Format for Knowledge Graph Completion
• Authors: Hung Nghiep Tran and Atsuhiro Takasu
• Abstract summary: Knowledge graph embedding methods perform the task by representing entities and relations as embedding vectors. Previous work has usually treated each embedding as a whole and has modeled the interactions between these whole embeddings. We propose the multi- partition embedding interaction (MEI) model with block term format to address this problem.
• Score: 3.718476964451589
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge graph completion is an important task that aims to predict the missing relational link between entities. Knowledge graph embedding methods perform this task by representing entities and relations as embedding vectors and modeling their interactions to compute the matching score of each triple. Previous work has usually treated each embedding as a whole and has modeled the interactions between these whole embeddings, potentially making the model excessively expensive or requiring specially designed interaction mechanisms. In this work, we propose the multi-partition embedding interaction (MEI) model with block term format to systematically address this problem. MEI divides each embedding into a multi-partition vector to efficiently restrict the interactions. Each local interaction is modeled with the Tucker tensor format and the full interaction is modeled with the block term tensor format, enabling MEI to control the trade-off between expressiveness and computational cost, learn the interaction mechanisms from data automatically, and achieve state-of-the-art performance on the link prediction task. In addition, we theoretically study the parameter efficiency problem and derive a simple empirically verified criterion for optimal parameter trade-off. We also apply the framework of MEI to provide a new generalized explanation for several specially designed interaction mechanisms in previous models. The source code is released at https://github.com/tranhungnghiep/MEI-KGE.

Related papers

• Relation Learning and Aggregate-attention for Multi-person Motion Prediction [13.052342503276936]
  Multi-person motion prediction considers not just the skeleton structures or human trajectories but also the interactions between others. Previous methods often overlook that the joints relations within an individual (intra-relation) and interactions among groups (inter-relation) are distinct types of representations. We introduce a new collaborative framework for multi-person motion prediction that explicitly modeling these relations.
  arXiv  Detail & Related papers  (2024-11-06T07:48:30Z)
• Matchmaker: Self-Improving Large Language Model Programs for Schema Matching [60.23571456538149]
  We propose a compositional language model program for schema matching, comprised of candidate generation, refinement and confidence scoring. Matchmaker self-improves in a zero-shot manner without the need for labeled demonstrations. Empirically, we demonstrate on real-world medical schema matching benchmarks that Matchmaker outperforms previous ML-based approaches.
  arXiv  Detail & Related papers  (2024-10-31T16:34:03Z)
• MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation [80.47072100963017]
  We introduce a novel and low-compute algorithm, Model Merging with Amortized Pareto Front (MAP) MAP efficiently identifies a set of scaling coefficients for merging multiple models, reflecting the trade-offs involved. We also introduce Bayesian MAP for scenarios with a relatively low number of tasks and Nested MAP for situations with a high number of tasks, further reducing the computational cost of evaluation.
  arXiv  Detail & Related papers  (2024-06-11T17:55:25Z)
• Composable Part-Based Manipulation [61.48634521323737]
  We propose composable part-based manipulation (CPM) to improve learning and generalization of robotic manipulation skills. CPM comprises a collection of composable diffusion models, where each model captures a different inter-object correspondence. We validate our approach in both simulated and real-world scenarios, demonstrating its effectiveness in achieving robust and generalized manipulation capabilities.
  arXiv  Detail & Related papers  (2024-05-09T16:04:14Z)
• Decomposing and Editing Predictions by Modeling Model Computation [75.37535202884463]
  We introduce a task called component modeling. The goal of component modeling is to decompose an ML model's prediction in terms of its components. We present COAR, a scalable algorithm for estimating component attributions.
  arXiv  Detail & Related papers  (2024-04-17T16:28:08Z)
• MEIM: Multi-partition Embedding Interaction Beyond Block Term Format for Efficient and Expressive Link Prediction [3.718476964451589]
  We introduce the Multi- Partition Embedding Interaction iMproved beyond block term format (MEIM) model. MEIM improves expressiveness while still being highly efficient, helping it to outperform strong baselines and achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2022-09-30T17:20:03Z)
• GraphFM: Graph Factorization Machines for Feature Interaction Modeling [27.307086868266012]
  We propose a novel approach, Graph Factorization Machine (GraphFM), by naturally representing features in the graph structure. In particular, we design a mechanism to select the beneficial feature interactions and formulate them as edges between features. The proposed model integrates the interaction function of FM into the feature aggregation strategy of Graph Neural Network (GNN)
  arXiv  Detail & Related papers  (2021-05-25T12:10:54Z)
• AutoDis: Automatic Discretization for Embedding Numerical Features in CTR Prediction [45.69943728028556]
  Learning sophisticated feature interactions is crucial for Click-Through Rate (CTR) prediction in recommender systems. Various deep CTR models follow an Embedding & Feature Interaction paradigm. We propose AutoDis, a framework that discretizes features in numerical fields automatically and is optimized with CTR models in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-12-16T14:31:31Z)
• AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding [40.900070190077024]
  We develop a novel Knowledge Graph Embedding (KGE) framework with Automated Entity TypE Representation (AutoETER) Our approach could model and infer all the relation patterns and complex relations. Experiments on four datasets demonstrate the superior performance of our model compared to state-of-the-art baselines on link prediction tasks.
  arXiv  Detail & Related papers  (2020-09-25T04:27:35Z)
• HittER: Hierarchical Transformers for Knowledge Graph Embeddings [85.93509934018499]
  We propose Hitt to learn representations of entities and relations in a complex knowledge graph. Experimental results show that Hitt achieves new state-of-the-art results on multiple link prediction. We additionally propose a simple approach to integrate Hitt into BERT and demonstrate its effectiveness on two Freebase factoid answering datasets.
  arXiv  Detail & Related papers  (2020-08-28T18:58:15Z)
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction [75.16836697734995]
  We propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS) AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service.
  arXiv  Detail & Related papers  (2020-03-25T06:53:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.