OpenHealth Lake: Designing and testing a data lakehouse platform for health applications

• URL: http://arxiv.org/abs/2605.19922v1
• Date: Tue, 19 May 2026 14:45:59 GMT
• Title: OpenHealth Lake: Designing and testing a data lakehouse platform for health applications
• Abstract summary: OpenHealth Lake is a data management prototype platform based on a data lakehouse architecture, data federation, and the FAIR principles.<n>The current prototype platform comprises a user-friendly website, an open API, Python and R packages, allowing users to interact with the platform in multiple ways.
• Score: 0.2539920413471809
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data management can be a complex challenge in fields such as bioinformatics and health sciences, which continuously generate extensive heterogeneous datasets. In the context of collaborative global health initiatives, secure storage and sharing of data are crucial to support impactful research. However, the absence of a unified data management platform complicates efficient data exchange and governance within these initiatives. In this paper, we introduce the design process of OpenHealth Lake, a data management prototype platform based on a data lakehouse architecture, data federation, and the FAIR principles. The platform is designed using open-source tools, guided by system requirements identified in previously published studies and complemented by insights from the existing literature. The current prototype platform comprises a user-friendly website, an open API, Python and R packages, allowing users to interact with the platform in multiple ways. Through a user study that included participants with varying technical backgrounds, we showed that our proposed data management prototype is both usable and useful. Our prototype design showcases the adaptability, scalability, and reproducibility of a lakehouse system that can be used by any organisation. It is designed as a flexible and complementary approach that allows organisations to customise data management systems to their specific requirements and resources, including cloud-based or self-hosted storage choices.

Related papers

• Project Imaging-X: A Survey of 1000+ Open-Access Medical Imaging Datasets for Foundation Model Development [314.80153557710616]
  We present the largest survey to date of medical image datasets, covering over 1,000 open-access datasets.<n>Our analysis exposes a landscape that is modest in scale, fragmented across narrowly scoped tasks, and unevenly distributed across organs and modalities.<n>We propose a metadata-driven fusion paradigm (MDFP) that integrates public datasets with shared modalities or tasks.
  arXiv  Detail & Related papers  (2026-03-29T00:46:53Z)
• A Framework and Prototype for a Navigable Map of Datasets in Engineering Design and Systems Engineering [0.0]
  This paper proposes a systematic framework for a "Map of Datasets in EDSE"<n>The framework is built upon a multi-dimensional taxonomy designed to classify engineering datasets by domain, lifecycle stage, data type, and format.<n>An analysis of the current data landscape reveals underrepresented areas ("data deserts") in early-stage design and system architecture, as well as relatively well-represented areas ("data oases") in predictive maintenance and autonomous systems.
  arXiv  Detail & Related papers  (2026-03-16T17:08:20Z)
• Data Science and Technology Towards AGI Part I: Tiered Data Management [53.64581824953229]
  We argue that the development of artificial intelligence is entering a new phase of data-model co-evolution.<n>We introduce an L0-L4 tiered data management framework, ranging from raw uncurated resources to organized and verifiable knowledge.<n>We validate the effectiveness of the proposed framework through empirical studies.
  arXiv  Detail & Related papers  (2026-02-09T18:47:51Z)
• AI-Powered Multi-Stakeholder Ecosystems for Global Development: A Design Research Study on the GSI D-Hub Proof-of-Concept Platform [0.0]
  This paper presents the Global Solutions Initiative (GSI) D-Hub, a data-driven coordination platform.<n>The system integrates structured data models, interpretable algorithms, and synthetic data pipelines to reduce information asymmetries.<n>Results show that explainable recommendations and contextual dashboards enhance trust, usability, and decision confidence.
  arXiv  Detail & Related papers  (2026-01-25T22:27:01Z)
• LLM-based Multi-Agent Blackboard System for Information Discovery in Data Science [69.1690891731311]
  We propose a novel multi-agent communication paradigm inspired by the blackboard architecture for traditional AI models.<n>In this framework, a central agent posts requests to a shared blackboard, and autonomous subordinate agents respond based on their capabilities.<n>We evaluate our method on three benchmarks that require explicit data discovery.
  arXiv  Detail & Related papers  (2025-09-30T22:34:23Z)
• Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models [83.65386456026441]
  Data-Juicer 2.0 is a data processing system backed by 100+ data processing operators spanning text, image, video, and audio modalities.<n>It supports more critical tasks including data analysis, synthesis, annotation, and foundation model post-training.<n>The system is publicly available and has been widely adopted in diverse research fields and real-world products such as Alibaba Cloud PAI.
  arXiv  Detail & Related papers  (2024-12-23T08:29:57Z)
• Architectural Design Decisions for Self-Serve Data Platforms in Data Meshes [3.627365672061558]
  Data mesh is an emerging decentralized approach to managing and generating value from analytical enterprise data at scale. It shifts the ownership of the data to the business domains closest to the data, promotes sharing and managing data as autonomous products, and uses a federated and automated data governance model. The data mesh relies on a managed data platform that offers services to domain and governance teams to build, share, and manage data products efficiently.
  arXiv  Detail & Related papers  (2024-02-07T09:13:26Z)
• A Versatile Data Fabric for Advanced IoT-Based Remote Health Monitoring [0.8789651809819904]
  This paper presents a data-centric and security-focused data fabric designed for digital health applications. The proposed data fabric comprises an architecture and a toolkit that facilitate the integration of heterogeneous data sources. We present the implementation of our data fabric in a home-based telemonitoring research project involving older adults.
  arXiv  Detail & Related papers  (2023-10-02T22:05:48Z)
• YMIR: A Rapid Data-centric Development Platform for Vision Applications [82.67319997259622]
  This paper introduces an open source platform for rapid development of computer vision applications. The platform puts the efficient data development at the center of the machine learning development process.
  arXiv  Detail & Related papers  (2021-11-19T05:02:55Z)
• CateCom: a practical data-centric approach to categorization of computational models [77.34726150561087]
  We present an effort aimed at organizing the landscape of physics-based and data-driven computational models. We apply object-oriented design concepts and outline the foundations of an open-source collaborative framework.
  arXiv  Detail & Related papers  (2021-09-28T02:59:40Z)
• INODE: Building an End-to-End Data Exploration System in Practice [Extended Vision] [30.411996388471817]
  INODE is an end-to-end data exploration system. We demonstrate it in three significant use cases in the fields of Cancer Biomarker Reearch, Research and Innovation Policy Making, and Astrophysics.
  arXiv  Detail & Related papers  (2021-04-09T05:04:04Z)
• Towards Compliant Data Management Systems for Healthcare ML [6.057289837472806]
  We review how data flows within machine learning projects in healthcare from source to storage to use in training algorithms and beyond. Our objective is to design tools to detect and track sensitive data across machines and users across the life cycle of a project. We build a prototype of the solution that demonstrates the difficulties in this domain.
  arXiv  Detail & Related papers  (2020-11-15T15:27:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.