Natural Language Knowledge Graph Query Execution: Leveraging Controlled Semantics in the LLM Context Window

• URL: http://arxiv.org/abs/2609.14652v1
• Date: Sun, 13 Sep 2026 16:36:40 GMT
• Title: Natural Language Knowledge Graph Query Execution: Leveraging Controlled Semantics in the LLM Context Window
• Abstract summary: We show that for database queries, data model concepts pass to LLMs more effectively through representations whose vocabulary terms carry declared, machine-readable semantics.<n> NLKGQ is a working system and reusable framework that does this for data modeled in a knowledge graph.<n>We evaluate on the DBLP-QuAD 2.0 benchmark (57.6% Match under deterministic re-scoring), DBLP-QuAD 3.1 (89.9% Match on 1,000 questions), SemOpenAlex (98% Match against a published baseline's 86% on the identical test set)
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Model (LLM) applications often transfer domain concepts into the model's context informally, through prompt prose, schema dumps, and examples. We show that for database queries, data model concepts pass to LLMs more effectively through representations whose vocabulary terms carry declared, machine-readable semantics (controlled semantics). NLKGQ is a working system and reusable framework that does this for data modeled in a knowledge graph. A formal OWL ontology serves as the transfer mechanism, concentrating the meaning of the data into semantically precise tokens the model can use directly. In a single LLM call, NLKGQ places in the context a system prompt instructing on SPARQL, the complete domain OWL ontology, and a domain-specific prompt addition, together with the user's natural language query. The model then generates the SPARQL query directly, zero-shot. Where the native vocabulary of an existing database or federation of databases is opaque, a wrapper ontology substitutes clean terms and a runtime rewriter restores the native forms. Evaluating on DBLP-QuAD 2.0 showed that its scores depend on the graph snapshot, the endpoint used, and the wording of its machine-generated questions, so we propose DBLP-QuAD 3.1, which maintains the intent of 2.0 while making reference results deterministic, revising reference SPARQL where needed, and rewriting the natural language questions, with a frontier model, to state each reference query's intent clearly and completely. We evaluate on the DBLP-QuAD 2.0 benchmark (57.6% Match under deterministic re-scoring), DBLP-QuAD 3.1 (89.9% Match on 1,000 questions), SemOpenAlex (98% Match against a published baseline's 86% on the identical test set), and neuroimaging metadata (100%).

Related papers

• Towards Researcher Agents for Knowledge-Graph Question Answering [0.0]
  We present an agentic text-to-SPARQL system that goes one step beyond static tool-using agents.<n>We instantiate the loop on DBpedia, evolve nine successive versions of the agent driven by a low-cost reasoning model, and deploy the best-performing configuration.
  arXiv  Detail & Related papers  (2026-08-07T18:42:53Z)
• Natural Language Access to Domain-Specific Metadata: A Reusable Framework for LLM Query Generation [0.0]
  We show that Large Language Models (LLMs) can generate accurate structured queries zero-shot without fine-tuning, retrieval augmentation, or multi-agent orchestration.<n>We present the Natural Language Knowledge Graph Query (NLKGQ) system, a framework and development process that enables natural language access to metadata in such archives.
  arXiv  Detail & Related papers  (2026-07-20T14:59:33Z)
• FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphs [0.5120567378386615]
  We propose a modular framework that supports fine-tuned LLMs as a core component, with optional context provided via RAG and a SPARQL query correction layer.<n>We measure query accuracy using BLEU and ROUGE metrics, and query result accuracy using relaxed exact match(RelaxedEM)<n> Experimental results demonstrate that fine-tuning achieves the highest overall performance, reaching 0.90 ROUGE-L for query accuracy and 0.85 RelaxedEM for result accuracy on the test set.
  arXiv  Detail & Related papers  (2025-08-14T09:08:50Z)
• Text-to-SPARQL Goes Beyond English: Multilingual Question Answering Over Knowledge Graphs through Human-Inspired Reasoning [51.203811759364925]
  mKGQAgent breaks down the task of converting natural language questions into SPARQL queries into modular, interpretable subtasks.<n> Evaluated on the DBpedia- and Corporate-based KGQA benchmarks within the Text2SPARQL challenge 2025, our approach took first place among the other participants.
  arXiv  Detail & Related papers  (2025-07-22T19:23:03Z)
• Effective Instruction Parsing Plugin for Complex Logical Query Answering on Knowledge Graphs [51.33342412699939]
  Knowledge Graph Query Embedding (KGQE) aims to embed First-Order Logic (FOL) queries in a low-dimensional KG space for complex reasoning over incomplete KGs. Recent studies integrate various external information (such as entity types and relation context) to better capture the logical semantics of FOL queries. We propose an effective Query Instruction Parsing (QIPP) that captures latent query patterns from code-like query instructions.
  arXiv  Detail & Related papers  (2024-10-27T03:18:52Z)
• Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA [51.3033125256716]
  We model the subgraph retrieval task as a conditional generation task handled by small language models. Our base generative subgraph retrieval model, consisting of only 220M parameters, competitive retrieval performance compared to state-of-the-art models. Our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks.
  arXiv  Detail & Related papers  (2024-10-08T15:22:36Z)
• UQE: A Query Engine for Unstructured Databases [71.49289088592842]
  We investigate the potential of Large Language Models to enable unstructured data analytics. We propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections.
  arXiv  Detail & Related papers  (2024-06-23T06:58:55Z)
• NL2KQL: From Natural Language to Kusto Query [1.7931930942711818]
  NL2KQL is an innovative framework that uses large language models (LLMs) to convert natural language queries (NLQs) to Kusto Query Language (KQL) queries.<n>To validate NL2KQL's performance, we utilize an array of online (based on query execution) and offline (based on query parsing) metrics.
  arXiv  Detail & Related papers  (2024-04-03T01:09:41Z)
• Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding [84.04706075621013]
  We present a general-purpose, modular neural semantic parsing framework based on token-level fine-grained query understanding. Our framework consists of three modules: named entity recognizer (NER), neural entity linker (NEL) and neural entity linker (NSP)
  arXiv  Detail & Related papers  (2022-09-28T21:00:30Z)
• Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing [66.55478402233399]
  We propose a framework to elicit relational structures via a probing procedure based on Poincar'e distance metric. Compared with commonly-used rule-based methods for schema linking, we found that probing relations can robustly capture semantic correspondences. Our framework sets new state-of-the-art performance on three benchmarks.
  arXiv  Detail & Related papers  (2022-06-28T14:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.