Related papers: Economics of Sourcing Human Data

Economics of Sourcing Human Data

URL: http://arxiv.org/abs/2502.07732v1
Date: Tue, 11 Feb 2025 17:51:52 GMT
Title: Economics of Sourcing Human Data
Authors: Sebastin Santy, Prasanta Bhattacharya, Manoel Horta Ribeiro, Kelsey Allen, Sewoong Oh,
Abstract summary: We argue that the widespread use of large language models threatens the quality and integrity of human-generated data. Existing data collection systems prioritize speed, scale, and efficiency at the cost of intrinsic human motivation. We propose that rethinking data collection systems to align with contributors' intrinsic motivations.
Score: 27.26816810619047
License:
Abstract: Progress in AI has relied on human-generated data, from annotator marketplaces to the wider Internet. However, the widespread use of large language models now threatens the quality and integrity of human-generated data on these very platforms. We argue that this issue goes beyond the immediate challenge of filtering AI-generated content--it reveals deeper flaws in how data collection systems are designed. Existing systems often prioritize speed, scale, and efficiency at the cost of intrinsic human motivation, leading to declining engagement and data quality. We propose that rethinking data collection systems to align with contributors' intrinsic motivations--rather than relying solely on external incentives--can help sustain high-quality data sourcing at scale while maintaining contributor trust and long-term participation.

Related papers

Data and System Perspectives of Sustainable Artificial Intelligence [43.21672481390316]
Sustainable AI is a subfield of AI for aiming to reduce environmental impact and achieve sustainability. In this article, we discuss current issues, opportunities and example solutions for addressing these issues.
arXiv Detail & Related papers (2025-01-13T17:04:23Z)
Data Quality Awareness: A Journey from Traditional Data Management to Data Science Systems [9.490118207943196]
We present a review of the evolution of data quality awareness from traditional data management systems to modern data-driven AI systems. As data science systems support a wide range of activities, our focus in this paper lies specifically in the analytics aspect driven by machine learning.
arXiv Detail & Related papers (2024-11-05T11:12:25Z)
Generative AI for Secure and Privacy-Preserving Mobile Crowdsensing [74.58071278710896]
generative AI has attracted much attention from both academic and industrial fields. Secure and privacy-preserving mobile crowdsensing (SPPMCS) has been widely applied in data collection/ acquirement.
arXiv Detail & Related papers (2024-05-17T04:00:58Z)
When AI Eats Itself: On the Caveats of AI Autophagy [18.641925577551557]
The AI autophagy phenomenon suggests a future where generative AI systems may increasingly consume their own outputs without discernment. This study examines the existing literature, delving into the consequences of AI autophagy, analyzing the associated risks, and exploring strategies to mitigate its impact.
arXiv Detail & Related papers (2024-05-15T13:50:23Z)
AI-Driven Frameworks for Enhancing Data Quality in Big Data Ecosystems: Error_Detection, Correction, and Metadata Integration [0.0]
This thesis proposes a novel set of interconnected frameworks aimed at enhancing big data quality comprehensively. Firstly, we introduce new quality metrics and a weighted scoring system for precise data quality assessment. Thirdly, we present a generic framework for detecting various quality anomalies using AI models.
arXiv Detail & Related papers (2024-05-06T21:36:45Z)
Best Practices and Lessons Learned on Synthetic Data [83.63271573197026]
The success of AI models relies on the availability of large, diverse, and high-quality datasets. Synthetic data has emerged as a promising solution by generating artificial data that mimics real-world patterns.
arXiv Detail & Related papers (2024-04-11T06:34:17Z)
Data-centric Artificial Intelligence: A Survey [47.24049907785989]
Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle.
arXiv Detail & Related papers (2023-03-17T17:44:56Z)
Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment [66.91538273487379]
There is a certain consensus about the need to develop AI applications with a Human-Centric approach. Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes. We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data.
arXiv Detail & Related papers (2023-02-13T16:44:44Z)
Data-centric AI: Perspectives and Challenges [51.70828802140165]
Data-centric AI (DCAI) advocates a fundamental shift from model advancements to ensuring data quality and reliability. We bring together three general missions: training data development, inference data development, and data maintenance.
arXiv Detail & Related papers (2023-01-12T05:28:59Z)
Trustworthy AI [75.99046162669997]
Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
arXiv Detail & Related papers (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.