Discovering Relationships in Data Lakes Using Large Language Models: An Industrial Case

• URL: http://arxiv.org/abs/2608.26750v1
• Date: Thu, 27 Aug 2026 07:44:39 GMT
• Title: Discovering Relationships in Data Lakes Using Large Language Models: An Industrial Case
• Abstract summary: ColRel builds column embeddings from metadata and data available at ingestion time.<n>Experiments on public benchmarks and an industrial ERP dataset show that ColRel is particularly effective in semantically related, weak-signal settings.
• Score: 0.5833117322405447
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data lakes rely on metadata to remain usable, yet this meta data is often limited or weakly informative for column relationship discovery, especially in ERP-derived datasets with coded or abbreviated schema labels. We propose ColRel, a two-stage method that builds column embeddings from metadata and data available at ingestion time. In difficult cases, such as coded schemata, business dictionaries help better interpret column names and support the generation of short natural-language descriptions used in the second stage. Experiments on public benchmarks and an industrial ERP dataset show that ColRel is particularly effective in semantically related, weak-signal settings.

Related papers

• TACO: Task-Aware Column Description Generation Using LLMs [21.74596076142191]
  We present TACO, a task-aware framework for automatic column description generation using large language models (LLMs)<n>Extensive experiments show that TACO consistently outperforms existing methods, improving downstream task performance by up to 32%.
  arXiv  Detail & Related papers  (2026-06-19T18:49:56Z)
• CAE: Character-Level Autoencoder for Non-Semantic Relational Data Grouping [0.9595254895337946]
  This paper introduces a novel Character-Level Autoencoder (CAE) approach that automatically identifies and groups semantically identical columns in non-semantic relational datasets.<n>Unlike conventional Natural Language Processing (NLP) models that struggle with limitations in semantic interpretability, our approach operates at the character level with fixed dictionary constraints.<n>By maintaining a fixed dictionary size, our method significantly reduces both memory requirements and training time, enabling efficient processing of large-scale industrial data environments.
  arXiv  Detail & Related papers  (2025-11-10T22:07:22Z)
• Schema Inference for Tabular Data Repositories Using Large Language Models [12.626848016550051]
  We present SI-LLM, which infers a concise conceptual schema for data using only column headers and cell values.<n> SI-LLM achieves promising end-to-end results, as well as better or comparable results to state-of-the-art methods at each step.
  arXiv  Detail & Related papers  (2025-09-04T19:50:16Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Large Language Models are Good Relational Learners [55.40941576497973]
  We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for large language models (LLMs)<n>Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to process and reason over complex entity relationships.
  arXiv  Detail & Related papers  (2025-06-06T04:07:55Z)
• UNJOIN: Enhancing Multi-Table Text-to-SQL Generation via Schema Simplification [50.59009084277447]
  We introduce UNJOIN, a framework that decouples the retrieval of schema elements from logic generation.<n>In the first stage, we merge the column names of all tables in the database into a single-table representation by prefixing each column with its table name.<n>In the second stage, the query is generated on this simplified schema and mapped back to the original schema by reconstructing JOINs, UNIONs, and relational logic.
  arXiv  Detail & Related papers  (2025-05-23T17:28:43Z)
• NameGuess: Column Name Expansion for Tabular Data [28.557115822407294]
  We introduce a new task, called NameGuess, to expand column names as a natural language generation problem. We create a training dataset of 384K abbreviated-expanded column pairs. We enhance auto-regressive language models by conditioning on table content and column header names.
  arXiv  Detail & Related papers  (2023-10-19T23:11:37Z)
• Matching Table Metadata with Business Glossaries Using Large Language Models [18.1687301652456]
  We study the problem of matching table metadata to a business glossary containing data labels and descriptions. The resulting matching enables the use of an available or curated business glossary for retrieval and analysis without or before requesting access to the data contents. We leverage the power of large language models (LLMs) to design generic matching methods that do not require manual tuning.
  arXiv  Detail & Related papers  (2023-09-08T02:23:59Z)
• Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization [80.94424037751243]
  In zero-shot multilingual extractive text summarization, a model is typically trained on English dataset and then applied on summarization datasets of other languages. We propose NLS (Neural Label Search for Summarization), which jointly learns hierarchical weights for different sets of labels together with our summarization model. We conduct multilingual zero-shot summarization experiments on MLSUM and WikiLingua datasets, and we achieve state-of-the-art results using both human and automatic evaluations.
  arXiv  Detail & Related papers  (2022-04-28T14:02:16Z)
• Deep Graph Matching and Searching for Semantic Code Retrieval [76.51445515611469]
  We propose an end-to-end deep graph matching and searching model based on graph neural networks. We first represent both natural language query texts and programming language code snippets with the unified graph-structured data. In particular, DGMS not only captures more structural information for individual query texts or code snippets but also learns the fine-grained similarity between them.
  arXiv  Detail & Related papers  (2020-10-24T14:16:50Z)
• DART: Open-Domain Structured Data Record to Text Generation [91.23798751437835]
  We present DART, an open domain structured DAta Record to Text generation dataset with over 82k instances (DARTs) We propose a procedure of extracting semantic triples from tables that encode their structures by exploiting the semantic dependencies among table headers and the table title. Our dataset construction framework effectively merged heterogeneous sources from open domain semantic parsing and dialogue-act-based meaning representation tasks.
  arXiv  Detail & Related papers  (2020-07-06T16:35:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.