Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing

• URL: http://arxiv.org/abs/2104.10097v1
• Date: Tue, 20 Apr 2021 16:30:59 GMT
• Title: Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing
• Authors: Boaz Shmueli, Jan Fell, Soumya Ray, Lun-Wei Ku
• Abstract summary: We find that the Final Rule, the common ethical framework used by researchers, did not anticipate the use of online crowdsourcing platforms for data collection. We enumerate common scenarios where crowdworkers performing NLP tasks are at risk of harm. We recommend that researchers evaluate these risks by considering the three ethical principles set up by the Belmont Report.
• Score: 7.148585157154561
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of crowdworkers in NLP research is growing rapidly, in tandem with the exponential increase in research production in machine learning and AI. Ethical discussion regarding the use of crowdworkers within the NLP research community is typically confined in scope to issues related to labor conditions such as fair pay. We draw attention to the lack of ethical considerations related to the various tasks performed by workers, including labeling, evaluation, and production. We find that the Final Rule, the common ethical framework used by researchers, did not anticipate the use of online crowdsourcing platforms for data collection, resulting in gaps between the spirit and practice of human-subjects ethics in NLP research. We enumerate common scenarios where crowdworkers performing NLP tasks are at risk of harm. We thus recommend that researchers evaluate these risks by considering the three ethical principles set up by the Belmont Report. We also clarify some common misconceptions regarding the Institutional Review Board (IRB) application. We hope this paper will serve to reopen the discussion within our community regarding the ethical use of crowdworkers.

Related papers

• Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models [53.316174782223115]
  This whitepaper offers an overview of the ethical considerations surrounding research into or with large language models (LLMs) As LLMs become more integrated into widely used applications, their societal impact increases, bringing important ethical questions to the forefront.
  arXiv  Detail & Related papers  (2024-10-17T18:36:02Z)
• What Can Natural Language Processing Do for Peer Review? [173.8912784451817]
  In modern science, peer review is widely used, yet it is hard, time-consuming, and prone to error. Since the artifacts involved in peer review are largely text-based, Natural Language Processing has great potential to improve reviewing. We detail each step of the process from manuscript submission to camera-ready revision, and discuss the associated challenges and opportunities for NLP assistance.
  arXiv  Detail & Related papers  (2024-05-10T16:06:43Z)
• Responsible AI Considerations in Text Summarization Research: A Review of Current Practices [89.85174013619883]
  We focus on text summarization, a common NLP task largely overlooked by the responsible AI community. We conduct a multi-round qualitative analysis of 333 summarization papers from the ACL Anthology published between 2020-2022. We focus on how, which, and when responsible AI issues are covered, which relevant stakeholders are considered, and mismatches between stated and realized research goals.
  arXiv  Detail & Related papers  (2023-11-18T15:35:36Z)
• Regulation and NLP (RegNLP): Taming Large Language Models [51.41095330188972]
  We argue how NLP research can benefit from proximity to regulatory studies and adjacent fields. We advocate for the development of a new multidisciplinary research space on regulation and NLP.
  arXiv  Detail & Related papers  (2023-10-09T09:22:40Z)
• The ethical ambiguity of AI data enrichment: Measuring gaps in research ethics norms and practices [2.28438857884398]
  This study explores how, and to what extent, comparable research ethics requirements and norms have developed for AI research and data enrichment. Leading AI venues have begun to establish protocols for human data collection, but these are are inconsistently followed by authors.
  arXiv  Detail & Related papers  (2023-06-01T16:12:55Z)
• Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing [45.72382504913193]
  We conduct a survey of NLP researchers and practitioners to understand the depth and their perspective of the problem. Based on the results of our survey, we offer a definition of dual use that is tailored to the needs of the NLP community. We discuss the current state and potential means for mitigating dual use in NLP and propose a checklist that can be integrated into existing conference ethics-frameworks.
  arXiv  Detail & Related papers  (2023-04-17T14:37:43Z)
• Use of Formal Ethical Reviews in NLP Literature: Historical Trends and Current Practices [6.195761193461355]
  Ethical aspects of research in language technologies have received much attention recently. It is a standard practice to get a study involving human subjects reviewed and approved by a professional ethics committee/board of the institution. With the rising concerns and discourse around the ethics of NLP, do we also observe a rise in formal ethical reviews of NLP studies?
  arXiv  Detail & Related papers  (2021-06-02T12:12:59Z)
• Case Study: Deontological Ethics in NLP [119.53038547411062]
  We study one ethical theory, namely deontological ethics, from the perspective of NLP. In particular, we focus on the generalization principle and the respect for autonomy through informed consent. We provide four case studies to demonstrate how these principles can be used with NLP systems.
  arXiv  Detail & Related papers  (2020-10-09T16:04:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.