Private, Augmentation-Robust and Task-Agnostic Data Valuation Approach for Data Marketplace

• URL: http://arxiv.org/abs/2411.00745v1
• Date: Fri, 01 Nov 2024 17:13:14 GMT
• Title: Private, Augmentation-Robust and Task-Agnostic Data Valuation Approach for Data Marketplace
• Authors: Tayyebeh Jahani-Nezhad, Parsa Moradi, Mohammad Ali Maddah-Ali, Giuseppe Caire,
• Abstract summary: PriArTa is an approach for computing the distance between the distribution of the buyer's existing dataset and the seller's dataset. PriArTa is communication-efficient, enabling the buyer to evaluate datasets without needing access to the entire dataset from each seller.
• Score: 56.78396861508909
• License:
• Abstract: Evaluating datasets in data marketplaces, where the buyer aim to purchase valuable data, is a critical challenge. In this paper, we introduce an innovative task-agnostic data valuation method called PriArTa which is an approach for computing the distance between the distribution of the buyer's existing dataset and the seller's dataset, allowing the buyer to determine how effectively the new data can enhance its dataset. PriArTa is communication-efficient, enabling the buyer to evaluate datasets without needing access to the entire dataset from each seller. Instead, the buyer requests that sellers perform specific preprocessing on their data and then send back the results. Using this information and a scoring metric, the buyer can evaluate the dataset. The preprocessing is designed to allow the buyer to compute the score while preserving the privacy of each seller's dataset, mitigating the risk of information leakage before the purchase. A key feature of PriArTa is its robustness to common data transformations, ensuring consistent value assessment and reducing the risk of purchasing redundant data. The effectiveness of PriArTa is demonstrated through experiments on real-world image datasets, showing its ability to perform privacy-preserving, augmentation-robust data valuation in data marketplaces.

Related papers

• A Survey on Data Markets [73.07800441775814]
  Growing trend of trading data for greater welfare has led to the emergence of data markets. A data market is any mechanism whereby the exchange of data products including datasets and data derivatives takes place. It serves as a coordinating mechanism by which several functions, including the pricing and the distribution of data, interact.
  arXiv  Detail & Related papers  (2024-11-09T15:09:24Z)
• Data Distribution Valuation [56.71023681599737]
  Existing data valuation methods define a value for a discrete dataset. In many use cases, users are interested in not only the value of the dataset, but that of the distribution from which the dataset was sampled. We propose a maximum mean discrepancy (MMD)-based valuation method which enables theoretically principled and actionable policies.
  arXiv  Detail & Related papers  (2024-10-06T07:56:53Z)
• Personalization of Dataset Retrieval Results using a Metadata-based Data Valuation Method [0.5999777817331317]
  We propose a novel data valuation method for a dataset retrieval use case in Ireland's National mapping agency. By leveraging metadata and a user's preferences, we estimate the personal value of each dataset. We validated the data value-based ranking against the stakeholders' ranking of the datasets.
  arXiv  Detail & Related papers  (2024-07-22T11:13:07Z)
• Truthful Dataset Valuation by Pointwise Mutual Information [28.63827288801458]
  We propose a new data valuation method that provably guarantees the following: data providers always maximize their expected score by truthfully reporting their observed data. Our method, following the paradigm of proper scoring rules, measures the pointwise mutual information (PMI) of the test dataset and the evaluated dataset.
  arXiv  Detail & Related papers  (2024-05-28T15:04:17Z)
• DAVED: Data Acquisition via Experimental Design for Data Markets [25.300193837833426]
  We propose a federated approach to the data acquisition problem that is inspired by linear experimental design. Our proposed data acquisition method achieves lower prediction error without requiring labeled validation data. The key insight of our work is that a method that directly estimates the benefit of acquiring data for test set prediction is particularly compatible with a decentralized market setting.
  arXiv  Detail & Related papers  (2024-03-20T18:05:52Z)
• Data Acquisition: A New Frontier in Data-centric AI [65.90972015426274]
  We first present an investigation of current data marketplaces, revealing lack of platforms offering detailed information about datasets. We then introduce the DAM challenge, a benchmark to model the interaction between the data providers and acquirers. Our evaluation of the submitted strategies underlines the need for effective data acquisition strategies in Machine Learning.
  arXiv  Detail & Related papers  (2023-11-22T22:15:17Z)
• A Survey of Data Pricing for Data Marketplaces [77.3189288320768]
  This paper attempts to comprehensively review the state-of-the-art on existing data pricing studies. Our key contribution lies in a new taxonomy of data pricing studies that unifies different attributes determining data prices.
  arXiv  Detail & Related papers  (2023-03-07T04:35:56Z)
• Fundamentals of Task-Agnostic Data Valuation [21.78555506720078]
  We study valuing the data of a data owner/seller for a data seeker/buyer. We focus on task-agnostic data valuation without any validation requirements.
  arXiv  Detail & Related papers  (2022-08-25T22:07:07Z)
• OSOUM Framework for Trading Data Research [79.0383470835073]
  We supply, to the best of our knowledge, the first open source simulation platform, Open SOUrce Market Simulator (OSOUM) to analyze trading markets and specifically data markets. We describe and implement a specific data market model, consisting of two types of agents: sellers who own various datasets available for acquisition, and buyers searching for relevant and beneficial datasets for purchase. Although commercial frameworks, intended for handling data markets, already exist, we provide a free and extensive end-to-end research tool for simulating possible behavior for both buyers and sellers participating in (data) markets.
  arXiv  Detail & Related papers  (2021-02-18T09:20:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.