Related papers: Type-Less yet Type-Aware Inductive Link Prediction with Pretrained Language Models

Type-Less yet Type-Aware Inductive Link Prediction with Pretrained Language Models

URL: http://arxiv.org/abs/2509.26224v1
Date: Tue, 30 Sep 2025 13:23:02 GMT
Title: Type-Less yet Type-Aware Inductive Link Prediction with Pretrained Language Models
Authors: Alessandro De Bellis, Salvatore Bufi, Giovanni Servedio, Vito Walter Anelli, Tommaso Di Noia, Eugenio Di Sciascio,
Abstract summary: We introduce TyleR, a Type-less yet type-awaRe approach for subgraph-based inductive link prediction.<n>We show that TyleR outperforms state-of-the-art baselines in scenarios with scarce type annotations and sparse graph connectivity.
Score: 47.6332883113148
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Inductive link prediction is emerging as a key paradigm for real-world knowledge graphs (KGs), where new entities frequently appear and models must generalize to them without retraining. Predicting links in a KG faces the challenge of guessing previously unseen entities by leveraging generalizable node features such as subgraph structure, type annotations, and ontological constraints. However, explicit type information is often lacking or incomplete. Even when available, type information in most KGs is often coarse-grained, sparse, and prone to errors due to human annotation. In this work, we explore the potential of pre-trained language models (PLMs) to enrich node representations with implicit type signals. We introduce TyleR, a Type-less yet type-awaRe approach for subgraph-based inductive link prediction that leverages PLMs for semantic enrichment. Experiments on standard benchmarks demonstrate that TyleR outperforms state-of-the-art baselines in scenarios with scarce type annotations and sparse graph connectivity. To ensure reproducibility, we share our code at https://github.com/sisinflab/tyler .

Related papers

Beyond Message Passing: Neural Graph Pattern Machine [50.78679002846741]
We introduce the Neural Graph Pattern Machine (GPM), a novel framework that bypasses message passing by learning directly from graph substructures.<n>GPM efficiently extracts, encodes, and prioritizes task-relevant graph patterns, offering greater expressivity and improved ability to capture long-range dependencies.
arXiv Detail & Related papers (2025-01-30T20:37:47Z)
Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
arXiv Detail & Related papers (2023-10-23T04:35:58Z)
Towards Few-shot Inductive Link Prediction on Knowledge Graphs: A Relational Anonymous Walk-guided Neural Process Approach [49.00753238429618]
Few-shot inductive link prediction on knowledge graphs aims to predict missing links for unseen entities with few-shot links observed. Recent inductive methods utilize the sub-graphs around unseen entities to obtain the semantics and predict links inductively. We propose a novel relational anonymous walk-guided neural process for few-shot inductive link prediction on knowledge graphs, denoted as RawNP.
arXiv Detail & Related papers (2023-06-26T12:02:32Z)
TypeT5: Seq2seq Type Inference using Static Analysis [51.153089609654174]
We present a new type inference method that treats type prediction as a code infilling task. Our method uses static analysis to construct dynamic contexts for each code element whose type signature is to be predicted by the model. We also propose an iterative decoding scheme that incorporates previous type predictions in the model's input context.
arXiv Detail & Related papers (2023-03-16T23:48:00Z)
Link Prediction on Latent Heterogeneous Graphs [18.110053023118294]
We study the challenging and unexplored problem of link prediction on a latent heterogeneous graph (LHG) We propose a model named LHGNN, based on the novel idea of semantic embedding at node and path levels, to capture latent semantics on and between nodes. We conduct extensive experiments on four benchmark datasets, and demonstrate the superior performance of LHGNN.
arXiv Detail & Related papers (2023-02-21T04:09:51Z)
Type-aware Embeddings for Multi-Hop Reasoning over Knowledge Graphs [18.56742938427262]
Multi-hop reasoning over real-life knowledge graphs (KGs) is a highly challenging problem. To address this problem, it has been recently introduced a promising approach based on jointly embedding logical queries and KGs. We propose a novel TypE-aware Message Passing (TEMP) model, which enhances the entity and relation representations in queries.
arXiv Detail & Related papers (2022-05-02T10:05:13Z)
Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference [28.78215056129358]
This work presents LITE, a new approach that formulates entity typing as a natural language inference (NLI) problem. Experiments show that, with limited training data, LITE obtains state-of-the-art performance on the UFET task.
arXiv Detail & Related papers (2022-02-12T23:56:26Z)
Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking [63.49779304362376]
Graph neural networks (GNNs) have become a popular approach to integrating structural inductive biases into NLP models. We introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges. We show that we can drop a large proportion of edges without deteriorating the performance of the model.
arXiv Detail & Related papers (2020-10-01T17:51:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.