Construction of Historical Knowledge Graphs Based on BERT and Graph Neural Networks

• URL: http://arxiv.org/abs/2606.01747v1
• Date: Mon, 01 Jun 2026 06:10:19 GMT
• Title: Construction of Historical Knowledge Graphs Based on BERT and Graph Neural Networks
• Abstract summary: This paper provides a high-level architecture that combines encoder bidirectional representations of transformers (BERT) and graph neural networks (GNN)<n>The texts of traditional history resolve linguistic ambiguities, references limited by context, and a lack of established grammatical norms in a systematic way.<n>The experiments make full use of a comprehensive collection of municipal records, parliamentary documents, and historical correspondence.
• Score: 6.470944338393257
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Through digital humanities research and scale-up historical data analysis, a significant amount of traditional historical text is converted into structured knowledge graphs. This paper provides a high-level architecture that combines bidirectional encoder representations of transformers (BERT) and graph neural networks (GNN) to extract the entities and relationships from various types of historical texts. The texts of traditional history resolve linguistic ambiguities, references limited by context, and a lack of established grammatical norms in a systematic way. This study develops a new image retrieval system based on FastRQNet and pre-trained vision-language model Vilt-qaformer+RoBInet in accordance with the aforementioned recommendations. The experiments make full use of a comprehensive collection of municipal records, parliamentary documents, and historical correspondence. When compared to conventional rule-based techniques and other popular deep-learning baselines, the joint BERT-GNN system obtains greater Precision, Recall, and F1-score (Table 2). Complex nested structures and implicit reference issues can be handled by this structure with sufficient accuracy and thoroughness when creating knowledge graphs. The aforementioned experiments show that combining relational graph learning algorithms with context-sensitive semantic representation techniques can automatically extract historical data to add accumulated wisdom to the knowledge repository.

Related papers

• KGPFN: Unlocking the Potential of Knowledge Graph Foundation Model via In-Context Learning [51.44698654440413]
  Knowledge graph (KG) foundation models aim to generalize across graphs with unseen entities and relations by learning transferable relational structure.<n>We propose KGPFN, a KG foundation model that unifies transferable relational regularities with inference-time in-context learning.<n>Experiments on 57 KG benchmarks demonstrate that KGPFN achieves strong adaptation to previously unseen graphs through in-context learning alone.
  arXiv  Detail & Related papers  (2026-05-14T14:41:19Z)
• Research on Graph-Retrieval Augmented Generation Based on Historical Text Knowledge Graphs [6.350401830141683]
  This article addresses domain knowledge gaps in general large language models for historical text analysis.<n>We propose the Graph RAG framework, combining chain-of-thought prompting, self-instruction generation, and process supervision.<n>This dataset supports automated historical knowledge extraction, reducing labor costs.
  arXiv  Detail & Related papers  (2025-06-18T08:20:29Z)
• Rule-Guided Joint Embedding Learning over Knowledge Graphs [2.797512394739081]
  We propose a novel model that integrates both contextual and textual signals into entity and relation embeddings.<n>To better utilize context, we introduce two metrics: confidence, computed via a rule-based method, and relatedness, derived from textual representations.
  arXiv  Detail & Related papers  (2023-12-01T19:58:31Z)
• CADGE: Context-Aware Dialogue Generation Enhanced with Graph-Structured Knowledge Aggregation [25.56539617837482]
  A novel context-aware graph-attention model (Context-aware GAT) is proposed. It assimilates global features from relevant knowledge graphs through a context-enhanced knowledge aggregation mechanism. Empirical results demonstrate that our framework outperforms conventional GNN-based language models in terms of performance.
  arXiv  Detail & Related papers  (2023-05-10T16:31:35Z)
• KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs [26.557447199727758]
  We propose a novel knowledge-aware language model framework based on fine-tuning process. Our model can efficiently incorporate world knowledge from KGs into existing language models such as BERT.
  arXiv  Detail & Related papers  (2021-09-09T12:39:17Z)
• GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [53.70520466556453]
  We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow. In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
  arXiv  Detail & Related papers  (2021-05-06T12:20:41Z)
• Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training [22.534866015730664]
  We verbalize the entire English Wikidata KG. We show that verbalizing a comprehensive, encyclopedic KG like Wikidata can be used to integrate structured KGs and natural language corpora.
  arXiv  Detail & Related papers  (2020-10-23T22:14:50Z)
• ENT-DESC: Entity Description Generation by Exploring Knowledge Graph [53.03778194567752]
  In practice, the input knowledge could be more than enough, since the output description may only cover the most significant knowledge. We introduce a large-scale and challenging dataset to facilitate the study of such a practical scenario in KG-to-text. We propose a multi-graph structure that is able to represent the original graph information more comprehensively.
  arXiv  Detail & Related papers  (2020-04-30T14:16:19Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)
• Improved Code Summarization via a Graph Neural Network [96.03715569092523]
  In general, source code summarization techniques use the source code as input and outputs a natural language description. We present an approach that uses a graph-based neural architecture that better matches the default structure of the AST to generate these summaries.
  arXiv  Detail & Related papers  (2020-04-06T17:36:42Z)
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs [96.73259297063619]
  We consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions. We leverage Generative Adrial Networks (GANs) to establish the connection between text and knowledge graph domain.
  arXiv  Detail & Related papers  (2020-01-08T01:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.