NeurDB: On the Design and Implementation of an AI-powered Autonomous Database

• URL: http://arxiv.org/abs/2408.03013v2
• Date: Sun, 05 Jan 2025 01:52:40 GMT
• Title: NeurDB: On the Design and Implementation of an AI-powered Autonomous Database
• Authors: Zhanhao Zhao, Shaofeng Cai, Haotian Gao, Hexiang Pan, Siqi Xiang, Naili Xing, Gang Chen, Beng Chin Ooi, Yanyan Shen, Yuncheng Wu, Meihui Zhang,
• Abstract summary: This paper introduces NeurDB, an AI-powered autonomous database. NeurDB deepens the fusion of AI and databases with adaptability to data and workload drift. Empirical evaluations demonstrate that NeurDB substantially outperforms existing solutions in managing AI analytics tasks.
• Score: 27.13518136879994
• License:
• Abstract: Databases are increasingly embracing AI to provide autonomous system optimization and intelligent in-database analytics, aiming to relieve end-user burdens across various industry sectors. Nonetheless, most existing approaches fail to account for the dynamic nature of databases, which renders them ineffective for real-world applications characterized by evolving data and workloads. This paper introduces NeurDB, an AI-powered autonomous database that deepens the fusion of AI and databases with adaptability to data and workload drift. NeurDB establishes a new in-database AI ecosystem that seamlessly integrates AI workflows within the database. This integration enables efficient and effective in-database AI analytics and fast-adaptive learned system components. Empirical evaluations demonstrate that NeurDB substantially outperforms existing solutions in managing AI analytics tasks, with the proposed learned components more effectively handling environmental dynamism than state-of-the-art approaches.

Related papers

• AnDB: Breaking Boundaries with an AI-Native Database for Universal Semantic Analysis [11.419119182421964]
  AnDB is an AI-native database that supports traditional O workloads and AI-driven tasks. AnDB allows users to perform semantic queries using intuitive-like statements without requiring AI expertise. AnDB future-proofs data management infrastructure, empowering users to effectively and efficiently harness the full potential of all kinds of data without starting from scratch.
  arXiv  Detail & Related papers  (2025-02-19T15:15:59Z)
• RelGNN: Composite Message Passing for Relational Deep Learning [56.48834369525997]
  We introduce RelGNN, a novel GNN framework specifically designed to capture the unique characteristics of relational databases. At the core of our approach is the introduction of atomic routes, which are sequences of nodes forming high-order tripartite structures. RelGNN consistently achieves state-of-the-art accuracy with up to 25% improvement.
  arXiv  Detail & Related papers  (2025-02-10T18:58:40Z)
• Top Ten Challenges Towards Agentic Neural Graph Databases [56.92578700681306]
  Graph databases (GDBs) like Neo4j and TigerGraph excel at handling interconnected data but lack advanced inference capabilities. This paper introduces Agentic Neural Graph Databases (Agentic NGDBs), which extend NGDBs with three core functionalities.
  arXiv  Detail & Related papers  (2025-01-24T04:06:50Z)
• ROMAS: A Role-Based Multi-Agent System for Database monitoring and Planning [11.589862354606476]
  We propose ROMAS, a Role-Based M ulti-A gent System designed to adapt to various scenarios while enabling low code development and one-click deployment. ROMAS has been effectively deployed in DB-GPT [Xue et al., 2023a, 2024b], a well-known project utilizing LLM-powered database analytics.
  arXiv  Detail & Related papers  (2024-12-18T05:45:39Z)
• LAMBDA: A Large Model Based Data Agent [7.240586338370509]
  We introduce LArge Model Based Data Agent (LAMBDA), a novel open-source, code-free multi-agent data analysis system. LAMBDA is designed to address data analysis challenges in complex data-driven applications. It has the potential to enhance data analysis paradigms by seamlessly integrating human and artificial intelligence.
  arXiv  Detail & Related papers  (2024-07-24T06:26:36Z)
• NeurDB: An AI-powered Autonomous Data System [44.14807794638682]
  We present NeurDB, an AI-powered autonomous data system designed to fully embrace AI design in each major system component. We outline the conceptual and architectural overview of NeurDB, discuss its design choices and key components, and report its current development and future plan.
  arXiv  Detail & Related papers  (2024-05-07T00:51:48Z)
• Powering In-Database Dynamic Model Slicing for Structured Data Analytics [31.360239181279525]
  We introduce LEADS, a novel dynamic model slicing technique to customize models for specifiedsql queries. LEADS improves the predictive modeling of structured data via the mixture of experts (MoE) and maintains efficiency by a SQL-aware gating network. Our experiments on real-world datasets demonstrate that LEADS consistently outperforms the baseline models.
  arXiv  Detail & Related papers  (2024-05-01T15:18:12Z)
• DNS-Rec: Data-aware Neural Architecture Search for Recommender Systems [79.76519917171261]
  This paper addresses the computational overhead and resource inefficiency prevalent in Sequential Recommender Systems (SRSs) We introduce an innovative approach combining pruning methods with advanced model designs. Our principal contribution is the development of a Data-aware Neural Architecture Search for Recommender System (DNS-Rec)
  arXiv  Detail & Related papers  (2024-02-01T07:22:52Z)
• Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming [77.38174112525168]
  We present Nemo, an end-to-end interactive Supervision system that improves overall productivity of WS learning pipeline by an average 20% (and up to 47% in one task) compared to the prevailing WS supervision approach.
  arXiv  Detail & Related papers  (2022-03-02T19:57:32Z)
• Unsupervised Domain Adaptive Learning via Synthetic Data for Person Re-identification [101.1886788396803]
  Person re-identification (re-ID) has gained more and more attention due to its widespread applications in video surveillance. Unfortunately, the mainstream deep learning methods still need a large quantity of labeled data to train models. In this paper, we develop a data collector to automatically generate synthetic re-ID samples in a computer game, and construct a data labeler to simultaneously annotate them.
  arXiv  Detail & Related papers  (2021-09-12T15:51:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.