PyTupli: A Scalable Infrastructure for Collaborative Offline Reinforcement Learning Projects

• URL: http://arxiv.org/abs/2505.16754v2
• Date: Fri, 23 May 2025 07:39:36 GMT
• Title: PyTupli: A Scalable Infrastructure for Collaborative Offline Reinforcement Learning Projects
• Authors: Hannah Markgraf, Michael Eichelbeck, Daria Cappey, Selin Demirtürk, Yara Schattschneider, Matthias Althoff,
• Abstract summary: offline reinforcement learning (RL) has gained traction as a powerful paradigm for learning control policies from pre-collected data.<n>PyTupli is a Python-based tool to streamline the creation, storage, and dissemination of benchmark environments.
• Score: 5.744272697629195
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Offline reinforcement learning (RL) has gained traction as a powerful paradigm for learning control policies from pre-collected data, eliminating the need for costly or risky online interactions. While many open-source libraries offer robust implementations of offline RL algorithms, they all rely on datasets composed of experience tuples consisting of state, action, next state, and reward. Managing, curating, and distributing such datasets requires suitable infrastructure. Although static datasets exist for established benchmark problems, no standardized or scalable solution supports developing and sharing datasets for novel or user-defined benchmarks. To address this gap, we introduce PyTupli, a Python-based tool to streamline the creation, storage, and dissemination of benchmark environments and their corresponding tuple datasets. PyTupli includes a lightweight client library with defined interfaces for uploading and retrieving benchmarks and data. It supports fine-grained filtering at both the episode and tuple level, allowing researchers to curate high-quality, task-specific datasets. A containerized server component enables production-ready deployment with authentication, access control, and automated certificate provisioning for secure use. By addressing key barriers in dataset infrastructure, PyTupli facilitates more collaborative, reproducible, and scalable offline RL research.

Related papers

• TaP: A Taxonomy-Guided Framework for Automated and Scalable Preference Data Generation [50.319535974012]
  Conducting supervised fine-tuning and preference fine-tuning on large language models (LLMs) requires high-quality datasets.<n>Most available datasets for supervised and preference fine-tuning are in English.<n>We propose the underlinetextbfTaxonomy-Guided underlinetextbfPreference Data Generation framework.
  arXiv  Detail & Related papers  (2025-06-30T15:45:28Z)
• Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models [64.28420991770382]
  Data-Juicer 2.0 is a data processing system backed by data processing operators spanning text, image, video, and audio modalities.<n>It supports more critical tasks including data analysis, annotation, and foundation model post-training.<n>It has been widely adopted in diverse research fields and real-world products such as Alibaba Cloud PAI.
  arXiv  Detail & Related papers  (2024-12-23T08:29:57Z)
• Cuvis.Ai: An Open-Source, Low-Code Software Ecosystem for Hyperspectral Processing and Classification [0.4038539043067986]
  cuvis.ai is an open-source and low-code software ecosystem for data acquisition, preprocessing, and model training. The package is written in Python and provides wrappers around common machine learning libraries.
  arXiv  Detail & Related papers  (2024-11-18T06:33:40Z)
• Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning [3.623224034411137]
  offline multi-agent reinforcement learning (MARL) is an exciting direction of research that uses static datasets to find optimal control policies for multi-agent systems. Though the field is by definition data-driven, efforts have thus far neglected data in their drive to achieve state-of-the-art results. We show how the majority of works generate their own datasets without consistent methodology and provide sparse information about the characteristics of these datasets.
  arXiv  Detail & Related papers  (2024-09-18T14:13:24Z)
• TorchRL: A data-driven decision-making library for PyTorch [20.776851077664915]
  PyTorch has ascended as a premier machine learning framework, yet it lacks a native and comprehensive library for decision and control tasks. We propose TorchRL, a generalistic control library for PyTorch that provides well-integrated, yet standalone components. We provide a detailed description of the building blocks and an extensive overview of the library across domains and tasks.
  arXiv  Detail & Related papers  (2023-06-01T11:45:45Z)
• Outsourcing Training without Uploading Data via Efficient Collaborative Open-Source Sampling [49.87637449243698]
  Traditional outsourcing requires uploading device data to the cloud server. We propose to leverage widely available open-source data, which is a massive dataset collected from public and heterogeneous sources. We develop a novel strategy called Efficient Collaborative Open-source Sampling (ECOS) to construct a proximal proxy dataset from open-source data for cloud training.
  arXiv  Detail & Related papers  (2022-10-23T00:12:18Z)
• DataPerf: Benchmarks for Data-Centric AI Development [81.03754002516862]
  DataPerf is a community-led benchmark suite for evaluating ML datasets and data-centric algorithms. We provide an open, online platform with multiple rounds of challenges to support this iterative development. The benchmarks, online evaluation platform, and baseline implementations are open source.
  arXiv  Detail & Related papers  (2022-07-20T17:47:54Z)
• PyRelationAL: a python library for active learning research and development [1.0061110876649197]
  Active learning (AL) is a sub-field of ML focused on the development of methods to iteratively and economically acquire data. Here, we introduce PyRelationAL, an open source library for AL research. We describe a modular toolkit based around a two step design methodology for composing pool-based active learning strategies.
  arXiv  Detail & Related papers  (2022-05-23T08:21:21Z)
• SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
  In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines. This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
  arXiv  Detail & Related papers  (2021-12-22T14:45:37Z)
• D4RL: Datasets for Deep Data-Driven Reinforcement Learning [119.49182500071288]
  We introduce benchmarks specifically designed for the offline setting, guided by key properties of datasets relevant to real-world applications of offline RL. By moving beyond simple benchmark tasks and data collected by partially-trained RL agents, we reveal important and unappreciated deficiencies of existing algorithms.
  arXiv  Detail & Related papers  (2020-04-15T17:18:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.