Beyond the Data Mesh Illusion: Designing Modern AI-augmented Lakehouses to Bridge the Gap Between Theory and Practice

• URL: http://arxiv.org/abs/2605.27131v1
• Date: Tue, 26 May 2026 15:00:57 GMT
• Title: Beyond the Data Mesh Illusion: Designing Modern AI-augmented Lakehouses to Bridge the Gap Between Theory and Practice
• Abstract summary: Enterprise data platforms face an enduring tension between domain self-service and holistic governance.<n>This paper argues that the flexibility-versus-control trade-off can be relaxed through an AI-augmented hub-and-spoke model layered on a modern lakehouse architecture.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Enterprise data platforms face an enduring tension between domain self-service and holistic governance. The data mesh paradigm proposed decentralized domain ownership as a remedy, but pure implementations frequently underdeliver: teams inherit new responsibilities without the platform maturity, tooling, or coordination mechanisms needed to exercise them effectively. This paper argues that the flexibility-versus-control trade-off can be relaxed through an AI-augmented hub-and-spoke model layered on a modern lakehouse architecture. A central hub (Center of Excellence) provides shared platform services, policy automation, and AI-enabled governance, automatically standardizing data products, generating quality rules, drafting data contracts, and reviewing changes for regressions. Domain spokes own business semantics, product backlogs, and local iteration cadence, progressively assuming greater responsibility as they mature. The same LLMs that automate governance tasks also lower the barrier for domain practitioners to develop genuine cross-functional expertise spanning business and data engineering, enabling spoke teams to take on greater end-to-end ownership without proportionally increasing their dependence on the hub. Natural-language conversational interfaces further democratize access for business users, exposing historically underutilized enterprise data. On the organizational side, we propose a staged framework that shifts ownership from hub to spokes, avoiding both centralized bottlenecks and uncoordinated decentralization. We evaluate the architecture through three outcome metrics: data product adoption, time-to-find, and time-to-insight, that tie platform success to measurable business value rather than internal activity.

Related papers

• Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges [58.887152635127165]
  Diffusion language models (DLMs) offer a non-autoregressive alternative for mobile edge agentic artificial intelligence (AI)<n>DLMs can update multiple uncertain tokens in parallel and exploit bidirectional context throughout the generation process.<n>This survey reviews DLM foundations and analyzes their suitability for edge settings under latency, memory, energy, bandwidth, privacy, and reliability constraints.
  arXiv  Detail & Related papers  (2026-09-04T06:16:49Z)
• Conversational Orchestration for Organic 6G [11.924983349269818]
  We propose a lightweight, decentralized conversational orchestration framework based on Large Language Model (LLM)-driven domain agents.<n>Each domain remains autonomous: an agent observes local state via tools, reasons in a closed loop, and exchanges summaries with neighboring agents over an Agent-to-Agent (A2A) overlay.<n>Fast feasible placement is enabled by periodic, routing-like dissemination of reachability advertisements.
  arXiv  Detail & Related papers  (2026-08-11T09:32:52Z)
• Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy [79.09714980186278]
  Building persistent embodied agents in unstructured environments requires unified orchestration of heterogeneous tools.<n>We present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces.<n>It achieves consistent improvements in end-to-end success across all complexity levels.
  arXiv  Detail & Related papers  (2026-06-25T16:36:35Z)
• Data-driven and distributed governance of building facilities management using decentralized autonomous organization, digital twin, and large language models [1.6767862854236946]
  This paper introduces a novel AI- and data-driven distributed governance framework for smart building management.<n>The framework integrates decentralized autonomous organizations (DAOs), digital twins, large language models (LLMs), and blockchain technology.
  arXiv  Detail & Related papers  (2026-04-16T20:10:35Z)
• Networking-Aware Energy Efficiency in Agentic AI Inference: A Survey [64.20429885919701]
  We propose an energy accounting framework identifying computational and communication costs across the Perception-Reasoning-Action cycle.<n>We establish a unified taxonomy spanning model simplification, computation control, input and attention optimization, and hardware-aware inference.<n>We identify open challenges of federated green learning, carbon-aware agency, 6th generation mobile communication (6G)-native Agentic AI, and self-sustaining systems.
  arXiv  Detail & Related papers  (2026-04-09T06:13:59Z)
• RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset [48.645870795753105]
  We introduce Robust Autonomous Data Acquisition for Robotics (RADAR)<n>RADAR is a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle.<n>In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-03-12T11:18:52Z)
• SoDA: An Efficient Interaction Paradigm for the Agentic Web [28.5099993831108]
  We define a future-oriented user sovereignty interaction paradigm, aiming to realize a fundamental shift from killing time to saving time.<n>Decoupling memory from application logic eliminates the structural basis of data lock-in.<n>Shifting from explicit manual instruction to implicit intent alignment resolves cognitive overload by offloading execution complexity.
  arXiv  Detail & Related papers  (2025-12-11T00:44:08Z)
• Trustless Federated Learning at Edge-Scale: A Compositional Architecture for Decentralized, Verifiable, and Incentive-Aligned Coordination [0.0]
  As federated learning matures, billions of edge devices holding sensitive data will be able to collectively improve models.<n>This democratic vision remains unrealized due to certain compositional gaps.<n>This work addresses these gaps by leveraging cryptographic receipts to prove aggregation correctness.
  arXiv  Detail & Related papers  (2025-11-26T07:13:38Z)
• Autonomy-Aware Clustering: When Local Decisions Supersede Global Prescriptions [0.3058685580689604]
  We introduce autonomy-aware clustering, a framework that learns and accounts for the influence of local autonomy without prior knowledge of its form.<n>We also propose the Adaptive Distance Estimation Network (ADEN), a transformer-based attention model that learns dependencies between entities and cluster representatives.
  arXiv  Detail & Related papers  (2025-09-30T04:44:36Z)
• Towards Human-Guided, Data-Centric LLM Co-Pilots [53.35493881390917]
  CliMB-DC is a human-guided, data-centric framework for machine learning co-pilots.<n>It combines advanced data-centric tools with LLM-driven reasoning to enable robust, context-aware data processing.<n>We show how CliMB-DC can transform uncurated datasets into ML-ready formats.
  arXiv  Detail & Related papers  (2025-01-17T17:51:22Z)
• Blockchain-enabled Trustworthy Federated Unlearning [50.01101423318312]
  Federated unlearning is a promising paradigm for protecting the data ownership of distributed clients. Existing works require central servers to retain the historical model parameters from distributed clients. This paper proposes a new blockchain-enabled trustworthy federated unlearning framework.
  arXiv  Detail & Related papers  (2024-01-29T07:04:48Z)
• Networked Communication for Decentralised Agents in Mean-Field Games [59.01527054553122]
  We introduce networked communication to the mean-field game framework.<n>We prove that our architecture has sample guarantees bounded between those of the centralised- and independent-learning cases.<n>We show that our networked approach has significant advantages over both alternatives in terms of robustness to update failures and to changes in population size.
  arXiv  Detail & Related papers  (2023-06-05T10:45:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.