Blue Data Intelligence Layer: Streaming Data and Agents for Multi-source Multi-modal Data-Centric Applications

• URL: http://arxiv.org/abs/2604.15233v1
• Date: Thu, 16 Apr 2026 17:10:21 GMT
• Title: Blue Data Intelligence Layer: Streaming Data and Agents for Multi-source Multi-modal Data-Centric Applications
• Abstract summary: NL2 systems aim to address the growing need for natural language interaction with data.<n>We present Blue's Data Intelligence Layer (DIL) designed to support multi-source, multi-modal, and data-centric applications.
• Score: 26.71604287422676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: NL2SQL systems aim to address the growing need for natural language interaction with data. However, real-world information rarely maps to a single SQL query because (1) users express queries iteratively (2) questions often span multiple data sources beyond the closed-world assumption of a single database, and (3) queries frequently rely on commonsense or external knowledge. Consequently, satisfying realistic data needs require integrating heterogeneous sources, modalities, and contextual data. In this paper, we present Blue's Data Intelligence Layer (DIL) designed to support multi-source, multi-modal, and data-centric applications. Blue is a compound AI system that orchestrates agents and data for enterprise settings. DIL serves as the data intelligence layer for agentic data processing, to bridge the semantic gap between user intent and available information by unifying structured enterprise data, world knowledge accessible through LLMs, and personal context obtained through interaction. At the core of DIL is a data registry that stores metadata for diverse data sources and modalities to enable both native and natural language queries. DIL treats LLMs, the Web, and the User as source 'databases', each with their own query interface, elevating them to first-class data sources. DIL relies on data planners to transform user queries into executable query plans. These plans are declarative abstractions that unify relational operators with other operators spanning multiple modalities. DIL planners support decomposition of complex requests into subqueries, retrieval from diverse sources, and finally reasoning and integration to produce final results. We demonstrate DIL through two interactive scenarios in which user queries dynamically trigger multi-source retrieval, cross-modal reasoning, and result synthesis, illustrating how compound AI systems can move beyond single database NL2SQL.

Related papers

• AskDB: An LLM Agent for Natural Language Interaction with Relational Databases [0.06524460254566904]
  We introduce AskDB, a large language model powered agent for interacting with databases through natural language.<n>AskDB supports both data analysis and administrative operations oversql databases through natural language.<n>Our results highlight the potential of AskDB as a unified and intelligent agent for relational database systems, offering an intuitive and accessible experience for end users.
  arXiv  Detail & Related papers  (2025-11-20T08:06:09Z)
• TEXT2DB: Integration-Aware Information Extraction with Large Language Model Agents [64.11547566154947]
  We propose a new formulation of IE TEXT2DB that emphasizes the integration of IE output and the target database.<n>We introduce a new benchmark featuring common demands such as data infilling, row population, and column addition.<n>Experiments show that OPAL can successfully adapt to diverse database schemas by generating different code plans and calling the required IE models.
  arXiv  Detail & Related papers  (2025-10-28T02:49:40Z)
• DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Search [61.77858432092777]
  We present DeepMMSearch-R1, the first multimodal large language model capable of performing on-demand, multi-turn web searches.<n>DeepMMSearch-R1 can initiate web searches based on relevant crops of the input image making the image search more effective.<n>We conduct extensive experiments across a range of knowledge-intensive benchmarks to demonstrate the superiority of our approach.
  arXiv  Detail & Related papers  (2025-10-14T17:59:58Z)
• LLM-based Multi-Agent Blackboard System for Information Discovery in Data Science [69.1690891731311]
  We propose a novel multi-agent communication paradigm inspired by the blackboard architecture for traditional AI models.<n>In this framework, a central agent posts requests to a shared blackboard, and autonomous subordinate agents respond based on their capabilities.<n>We evaluate our method on three benchmarks that require explicit data discovery.
  arXiv  Detail & Related papers  (2025-09-30T22:34:23Z)
• DBRouting: Routing End User Queries to Databases for Answerability [11.6206961865716]
  We define the novel task of routing an end-user query to the appropriate data-source, where the data-sources are databases.<n>We create baselines on datasets by using open-source LLMs, using both pre-trained and task specific embeddings fine-tuned using the training data.<n>We find that the task becomes more difficult with an increase in the number of data-sources.
  arXiv  Detail & Related papers  (2025-01-27T17:09:47Z)
• Explainable Multi-Modal Data Exploration in Natural Language via LLM Agent [6.147666891384964]
  XMODE is a system that enables explainable, multi-modal data exploration in natural language.<n>XMODE is inspired by a real-world use case that enables users to explore multi-modal information systems.
  arXiv  Detail & Related papers  (2024-12-24T13:42:44Z)
• Relational Database Augmented Large Language Model [59.38841050766026]
  Large language models (LLMs) excel in many natural language processing (NLP) tasks. They can only incorporate new knowledge through training or supervised fine-tuning processes. This precise, up-to-date, and private information is typically stored in relational databases.
  arXiv  Detail & Related papers  (2024-07-21T06:19:10Z)
• UQE: A Query Engine for Unstructured Databases [71.49289088592842]
  We investigate the potential of Large Language Models to enable unstructured data analytics. We propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections.
  arXiv  Detail & Related papers  (2024-06-23T06:58:55Z)
• IQLS: Framework for leveraging Metadata to enable Large Language Model based queries to complex, versatile Data [0.20482269513546458]
  The Intelligent Query and Learning System (IQLS) simplifies the process by allowing natural language use to simplify data retrieval. It maps structured data into a framework based on the available metadata and available data models. The IQLS enables the agent to fulfill tasks given by the user query through interfaces.
  arXiv  Detail & Related papers  (2024-05-04T13:44:05Z)
• Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub [79.31134731122462]
  We introduce OpenAct benchmark to evaluate the open-domain task-solving capability, built on human expert consultation and repositories in GitHub.<n>We present OpenAgent, a novel LLM-based agent system that can tackle evolving queries in open domains through autonomously integrating specialized tools from GitHub.
  arXiv  Detail & Related papers  (2023-12-28T15:47:30Z)
• Querying Large Language Models with SQL [16.383179496709737]
  In many use-cases, information is stored in text but not available in structured data. With the rise of pre-trained Large Language Models (LLMs), there is now an effective solution to store and use information extracted from massive corpora of text documents. We present Galois, a prototype based on a traditional database architecture, but with new physical operators for querying the underlying LLM.
  arXiv  Detail & Related papers  (2023-04-02T06:58:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.