Schema-First Retrieval: Embedding Catalogs for Natural Language Analytics

• URL: http://arxiv.org/abs/2606.28387v1
• Date: Tue, 23 Jun 2026 02:20:36 GMT
• Title: Schema-First Retrieval: Embedding Catalogs for Natural Language Analytics
• Abstract summary: Modern warehouses contain thousands of tables, abbreviated columns, informal metrics, hidden join conventions, and permission boundaries not captured by raw table names.<n>We introduce Enterprise-First Retrieval, a retrieval layer that embeds catalog metadata rather than warehouse rows.<n>The system indexes five typed catalog objects, tables, columns, metrics, relationships, and query history, using object-specific text templates.
• Score: 0.10210859604701106
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Enterprise text-to-SQL systems often fail before SQL is generated: the model receives the wrong schema context. Modern warehouses contain thousands of tables, abbreviated columns, informal metrics, hidden join conventions, and permission boundaries that are not captured by raw table names. We introduce Schema-First Retrieval, a retrieval layer that embeds catalog metadata rather than warehouse rows. The system indexes five typed catalog objects, tables, columns, metrics, relationships, and query history, using object-specific text templates. At query time, it combines parallel vector search, lineage expansion, cross-encoder reranking, workload memory, and deterministic access-control gates before SQL generation. On CRUSH4SQL (1,534 questions), Schema-First Retrieval reaches 96.4% table recall@20 and cross-encoder reranking adds +11.1 points at column recall@10; against an equally-templated BM25 baseline, semantic retrieval is +32.8 points at table recall@5. On SEDE (857 questions), query history raises table recall@5 from 52.1% to 92.3%. On BIRD (96 questions), schema-first context reduces SQL execution errors from 15.6% to 6.2%, a 2.5x reduction. These results show that catalog selection is a first-class retrieval problem for natural language analytics, not a prompt formatting detail.

Related papers

• AutoSQL: Extracting SQL Templates from Imperative ORM Code in Large-Scale Repositories [43.381882752672446]
  We present Auto sequences, a system that reconstructssql templates from Go ORM code.<n>We show that Auto achieves 68.04% to 72.18% recall, exceeding the static reachability baseline by 11.80% to 15.94% and outperforming existing methods by 8.52% to 21.50%.
  arXiv  Detail & Related papers  (2026-08-16T07:32:53Z)
• Finding the Right Tables and Columns: A Benchmark and Corpus-Adaptive Embeddings for SQL Schema Retrieval [9.741583305127994]
  We recast five text-to-language datasets as retrieval tasks at both table and column granularity.<n>We show that off-the-shelf text and code embedders transfer poorly to this setting.<n>We propose corpus-adaptive fine-tuning: natural-language queries are synthesized directly from the target corpus.
  arXiv  Detail & Related papers  (2026-07-14T22:31:55Z)
• Querying an astronomical database using large language models: the ALeRCE text-to-SQL system [0.6160458642720921]
  We develop a text-to-context (structured query language) system based on large language models (LLMs)<n>We apply it to the Automatic Learning for the Rapid Classification of Events (ALeRCE) astronomical database.<n>ALeRCE is a community broker for the Zwicky Transient Facility and the Vera C. Rubin Observatory.
  arXiv  Detail & Related papers  (2026-06-16T16:12:16Z)
• CYGNET: Cypher Gate for Neural Execution Triage and Cost Containment [0.0]
  We place a pre-execution gate between query generation and a production Neo4j database.<n>The gate validates structure through a four-backend chain culminating in execution against a mirror graph at 5.6 ms median latency.<n>On seven CypherBench schemas (2348 questions, ACL 2025) the pipeline maintains generation accuracy on every model tested, confirming it operates as a safe defensive layer.
  arXiv  Detail & Related papers  (2026-06-03T09:13:05Z)
• Schema-Aware Localisation (SAL): Live Schema Grounding and Hallucination Validation for Oracle NL2SQL [0.19116784879310023]
  Columns and aliases are hallucinated and dialect-specific syntax is missed, leading to ORA-00904 invalid-identifier errors.<n>This paper introduces-Aware Localisation (SAL), a lightweight layer for Oracle NL2 that requires no model retraining.<n>We evaluate SAL on 500 TPC-H natural language questions executed against a live Oracle Autonomous Database 23c instance.
  arXiv  Detail & Related papers  (2026-06-03T03:12:18Z)
• CORE-T: COherent REtrieval of Tables for Text-to-SQL [91.76918495375384]
  CORE-T is a scalable, training-free framework that enriches tables with purpose metadata and pre-computes a lightweight table-compatibility cache.<n>Across Bird, Spider, and MMQA, CORE-T improves table-selection F1 by up to 22.7 points while retrieving up to 42% fewer tables.
  arXiv  Detail & Related papers  (2026-01-19T14:51:23Z)
• AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale [68.29002818359844]
  Existing methods incur prohibitive costs, struggle to trade off recall and noise, and scale poorly to large databases.<n>We present textbfAutoLink, an autonomous agent framework that reformulates schema linking as an iterative, agent-driven process.<n>Experiments demonstrate AutoLink's superior performance, achieving state-of-the-art strict schema linking recall, i.e., textbf68.7% EX on Bird-Dev (better than CHESS) and textbf34.9% EX on Spider-2.0-Lite (ranking 2nd
  arXiv  Detail & Related papers  (2025-11-21T12:12:17Z)
• UNJOIN: Enhancing Multi-Table Text-to-SQL Generation via Schema Simplification [50.59009084277447]
  We introduce UNJOIN, a framework that decouples the retrieval of schema elements from logic generation.<n>In the first stage, we merge the column names of all tables in the database into a single-table representation by prefixing each column with its table name.<n>In the second stage, the query is generated on this simplified schema and mapped back to the original schema by reconstructing JOINs, UNIONs, and relational logic.
  arXiv  Detail & Related papers  (2025-05-23T17:28:43Z)
• RSL-SQL: Robust Schema Linking in Text-to-SQL Generation [51.00761167842468]
  We propose a novel framework called RSL- that combines bidirectional schema linking, contextual information augmentation, binary selection strategy, and multi-turn self-correction. benchmarks demonstrate that our approach achieves SOTA execution accuracy among open-source solutions, with 67.2% on BIRD and 87.9% on GPT-4ocorrection. Our approach outperforms a series of GPT-4 based Text-to-Seek systems when adopting DeepSeek (much cheaper) with same intact prompts.
  arXiv  Detail & Related papers  (2024-10-31T16:22:26Z)
• SQL-to-Schema Enhances Schema Linking in Text-to-SQL [15.6857201570992]
  In text-to-speech methods, there is a need to filter out unnecessary tables and columns. Previous approaches have involved sorting tables and columns based on their relevance to the question. We propose an inventive schema linking method in two steps.
  arXiv  Detail & Related papers  (2024-05-15T12:22:48Z)
• Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval [52.592071689901196]
  We introduce a method that uncovers useful join relations for any query and database during table retrieval.<n>Our method outperforms the state-of-the-art approaches for table retrieval by up to 9.3% in F1 score and for end-to-end QA by up to 5.4% in accuracy.
  arXiv  Detail & Related papers  (2024-04-15T15:55:01Z)
• MURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQL [51.48239006107272]
  Multi-hop table retrieval with removal (MURRE) removes previously retrieved information from the question to guide towards unretrieved relevant tables. Experiments on two open-domain text-to- retriever datasets demonstrate an average improvement of 5.7% over the previous state-of-the-art results.
  arXiv  Detail & Related papers  (2024-02-16T13:14:35Z)
• Analyzing the Effectiveness of Large Language Models on Text-to-SQL Synthesis [4.412170175171256]
  This study investigates various approaches to using Large Language Models for Text-to- program synthesis. The goal was to input a natural language question along with the database schema and output the correct SELECT query.
  arXiv  Detail & Related papers  (2024-01-22T22:05:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.