LakeMLB: Data Lake Machine Learning Benchmark

• URL: http://arxiv.org/abs/2602.10441v1
• Date: Wed, 11 Feb 2026 02:33:29 GMT
• Title: LakeMLB: Data Lake Machine Learning Benchmark
• Authors: Feiyu Pan, Tianbin Zhang, Aoqian Zhang, Yu Sun, Zheng Wang, Lixing Chen, Li Pan, Jianhua Li,
• Abstract summary: We present LakeMLB (Data Lake Machine Learning Benchmark), designed for the most common multi-source, multi-table scenarios in data lakes.<n>LakeMLB focuses on two representative multi-table scenarios, Union and Join, and provides three real-world datasets for each scenario, covering government open data, finance, Wikipedia, and online marketplaces.
• Score: 15.634664259138157
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern data lakes have emerged as foundational platforms for large-scale machine learning, enabling flexible storage of heterogeneous data and structured analytics through table-oriented abstractions. Despite their growing importance, standardized benchmarks for evaluating machine learning performance in data lake environments remain scarce. To address this gap, we present LakeMLB (Data Lake Machine Learning Benchmark), designed for the most common multi-source, multi-table scenarios in data lakes. LakeMLB focuses on two representative multi-table scenarios, Union and Join, and provides three real-world datasets for each scenario, covering government open data, finance, Wikipedia, and online marketplaces. The benchmark supports three representative integration strategies: pre-training-based, data augmentation-based, and feature augmentation-based approaches. We conduct extensive experiments with state-of-the-art tabular learning methods, offering insights into their performance under complex data lake scenarios. We release both datasets and code to facilitate rigorous research on machine learning in data lake ecosystems; the benchmark is available at https://github.com/zhengwang100/LakeMLB.

Related papers

• LLM-based Multi-Agent Blackboard System for Information Discovery in Data Science [69.1690891731311]
  We propose a novel multi-agent communication paradigm inspired by the blackboard architecture for traditional AI models.<n>In this framework, a central agent posts requests to a shared blackboard, and autonomous subordinate agents respond based on their capabilities.<n>We evaluate our method on three benchmarks that require explicit data discovery.
  arXiv  Detail & Related papers  (2025-09-30T22:34:23Z)
• TAIJI: MCP-based Multi-Modal Data Analytics on Data Lakes [25.05627023905607]
  We envision a new multi-modal data analytics system based on the Model Context Protocol (MCP)<n>First, we define a semantic operator hierarchy tailored for querying multi-modal data in data lakes.<n>Next, we introduce an MCP-based execution framework, in which each MCP server hosts specialized foundation models optimized for specific data modalities.
  arXiv  Detail & Related papers  (2025-05-16T14:03:30Z)
• BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data [61.936320820180875]
  Large language models (LLMs) have become increasingly pivotal across various domains. BabelBench is an innovative benchmark framework that evaluates the proficiency of LLMs in managing multimodal multistructured data with code execution. Our experimental findings on BabelBench indicate that even cutting-edge models like ChatGPT 4 exhibit substantial room for improvement.
  arXiv  Detail & Related papers  (2024-10-01T15:11:24Z)
• Stalactite: Toolbox for Fast Prototyping of Vertical Federated Learning Systems [37.11550251825938]
  We present emphStalactite - an open-source framework for Vertical Federated Learning (VFL) systems. VFL is a type of FL where data samples are divided by features across several data owners. We demonstrate its use on a real-world recommendation datasets.
  arXiv  Detail & Related papers  (2024-09-23T21:29:03Z)
• Retrieve, Merge, Predict: Augmenting Tables with Data Lakes [7.449868392714658]
  We present an in-depth analysis of automated table augmentation for machine learning tasks.<n>We analyze different methods for the three main steps: retrieving joinable tables, merging information, and predicting with the resultant table.<n>We use two data lakes: Open Data US, a well-referenced real data lake, and a novel semi-synthetic dataset, YADL (Yet Another Data Lake)
  arXiv  Detail & Related papers  (2024-02-09T09:48:38Z)
• Relational Deep Learning: Graph Representation Learning on Relational Databases [69.7008152388055]
  We introduce an end-to-end representation approach to learn on data laid out across multiple tables. Message Passing Graph Neural Networks can then automatically learn across the graph to extract representations that leverage all data input.
  arXiv  Detail & Related papers  (2023-12-07T18:51:41Z)
• Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning [11.205441416962284]
  We introduce dataset grouper, a library to create large-scale group-structured datasets. It enables federated learning simulation at the scale of foundation models.
  arXiv  Detail & Related papers  (2023-07-18T20:27:45Z)
• FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings [51.09574369310246]
  Federated Learning (FL) is a novel approach enabling several clients holding sensitive data to collaboratively train machine learning models. We propose a novel cross-silo dataset suite focused on healthcare, FLamby, to bridge the gap between theory and practice of cross-silo FL. Our flexible and modular suite allows researchers to easily download datasets, reproduce results and re-use the different components for their research.
  arXiv  Detail & Related papers  (2022-10-10T12:17:30Z)
• Deep Lake: a Lakehouse for Deep Learning [0.0]
  Deep Lake is an open-source lakehouse for deep learning applications developed at Activeloop. This paper presents Deep Lake, an open-source lakehouse for deep learning applications developed at Activeloop.
  arXiv  Detail & Related papers  (2022-09-22T05:04:09Z)
• TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments [84.6017003787244]
  This work proposes a synthetic data generation pipeline to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation.
  arXiv  Detail & Related papers  (2022-08-16T20:46:08Z)
• Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments [89.04823188871906]
  Generation of diverse realistic scenarios is challenging for real-time strategy (RTS) environments. Most of the existing simulators rely on randomly generating the environments. We introduce the benefits of adopting an existing formal scenario specification language, SCENIC, to assist researchers.
  arXiv  Detail & Related papers  (2021-06-18T21:49:46Z)
• Open Graph Benchmark: Datasets for Machine Learning on Graphs [86.96887552203479]
  We present the Open Graph Benchmark (OGB) to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multiple important graph ML tasks, and cover a diverse range of domains. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics.
  arXiv  Detail & Related papers  (2020-05-02T03:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.