Trust in Data Science: Collaboration, Translation, and Accountability in
Corporate Data Science Projects
- URL: http://arxiv.org/abs/2002.03389v1
- Date: Sun, 9 Feb 2020 15:50:50 GMT
- Title: Trust in Data Science: Collaboration, Translation, and Accountability in
Corporate Data Science Projects
- Authors: Samir Passi, Steven J. Jackson
- Abstract summary: We describe four common tensions in applied data science work: (un)equivocal numbers, (counter)intuitive knowledge, (in)credible data, and (in)scrutable models.
We show how organizational actors establish and re-negotiate trust under messy and uncertain analytic conditions through practices of skepticism, assessment, and credibility.
We conclude by discussing the implications of our findings for data science research and practice, both within and beyond CSCW.
- Score: 6.730787776951012
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The trustworthiness of data science systems in applied and real-world
settings emerges from the resolution of specific tensions through situated,
pragmatic, and ongoing forms of work. Drawing on research in CSCW, critical
data studies, and history and sociology of science, and six months of immersive
ethnographic fieldwork with a corporate data science team, we describe four
common tensions in applied data science work: (un)equivocal numbers,
(counter)intuitive knowledge, (in)credible data, and (in)scrutable models. We
show how organizational actors establish and re-negotiate trust under messy and
uncertain analytic conditions through practices of skepticism, assessment, and
credibility. Highlighting the collaborative and heterogeneous nature of
real-world data science, we show how the management of trust in applied
corporate data science settings depends not only on pre-processing and
quantification, but also on negotiation and translation. We conclude by
discussing the implications of our findings for data science research and
practice, both within and beyond CSCW.
Related papers
- MMSci: A Multimodal Multi-Discipline Dataset for PhD-Level Scientific Comprehension [59.41495657570397]
We collected a multimodal, multidisciplinary dataset from open-access scientific articles published in Nature Communications journals.
This dataset spans 72 scientific disciplines, ensuring both diversity and quality.
We created benchmarks with various tasks and settings to comprehensively evaluate LMMs' capabilities in understanding scientific figures and content.
arXiv Detail & Related papers (2024-07-06T00:40:53Z) - Data-Centric AI in the Age of Large Language Models [51.20451986068925]
This position paper proposes a data-centric viewpoint of AI research, focusing on large language models (LLMs)
We make the key observation that data is instrumental in the developmental (e.g., pretraining and fine-tuning) and inferential stages (e.g., in-context learning) of LLMs.
We identify four specific scenarios centered around data, covering data-centric benchmarks and data curation, data attribution, knowledge transfer, and inference contextualization.
arXiv Detail & Related papers (2024-06-20T16:34:07Z) - Data Science for Social Good [2.8621556092850065]
We present a framework for "data science for social good" (DSSG) research.
We perform an analysis of the literature to empirically demonstrate the paucity of work on DSSG in information systems.
We hope that this article and the special issue will spur future DSSG research.
arXiv Detail & Related papers (2023-11-02T15:40:20Z) - On Responsible Machine Learning Datasets with Fairness, Privacy, and
Regulatory Norms [58.93352076927003]
There have been severe concerns over the trustworthiness of AI technologies.
Machine and deep learning algorithms depend heavily on the data used during their development.
We propose a framework to evaluate the datasets through a responsible rubric.
arXiv Detail & Related papers (2023-10-24T14:01:53Z) - Auditing and Generating Synthetic Data with Controllable Trust Trade-offs [54.262044436203965]
We introduce a holistic auditing framework that comprehensively evaluates synthetic datasets and AI models.
It focuses on preventing bias and discrimination, ensures fidelity to the source data, assesses utility, robustness, and privacy preservation.
We demonstrate the framework's effectiveness by auditing various generative models across diverse use cases.
arXiv Detail & Related papers (2023-04-21T09:03:18Z) - Assessing Scientific Contributions in Data Sharing Spaces [64.16762375635842]
This paper introduces the SCIENCE-index, a blockchain-based metric measuring a researcher's scientific contributions.
To incentivize researchers to share their data, the SCIENCE-index is augmented to include a data-sharing parameter.
Our model is evaluated by comparing the distribution of its output for geographically diverse researchers to that of the h-index.
arXiv Detail & Related papers (2023-03-18T19:17:47Z) - TAPS Responsibility Matrix: A tool for responsible data science by
design [2.2973034509761816]
We describe the Transparency, Accountability, Privacy, and Societal Responsibility Matrix (TAPS-RM) as framework to explore social, legal, and ethical aspects of data science projects.
We map the developed model of TAPS-RM with well-known initiatives for open data.
We conclude that TAPS-RM is a tool to reflect on responsibilities at a data science project level and can be used to advance responsible data science by design.
arXiv Detail & Related papers (2023-02-02T12:09:14Z) - Modeling Information Change in Science Communication with Semantically
Matched Paraphrases [50.67030449927206]
SPICED is the first paraphrase dataset of scientific findings annotated for degree of information change.
SPICED contains 6,000 scientific finding pairs extracted from news stories, social media discussions, and full texts of original papers.
Models trained on SPICED improve downstream performance on evidence retrieval for fact checking of real-world scientific claims.
arXiv Detail & Related papers (2022-10-24T07:44:38Z) - SciTweets -- A Dataset and Annotation Framework for Detecting Scientific
Online Discourse [2.3371548697609303]
Scientific topics, claims and resources are increasingly debated as part of online discourse.
This has led to both significant societal impact and increased interest in scientific online discourse from various disciplines.
Research across disciplines currently suffers from a lack of robust definitions of the various forms of science-relatedness.
arXiv Detail & Related papers (2022-06-15T08:14:55Z) - Model Positionality and Computational Reflexivity: Promoting Reflexivity
in Data Science [10.794642538442107]
We describe how the concepts of positionality and reflexivity can be adapted to provide a framework for understanding data science work.
We describe the challenges of adapting these concepts for data science work and offer annotator fingerprinting and position mining as promising solutions.
arXiv Detail & Related papers (2022-03-08T16:02:03Z) - A survey study of success factors in data science projects [0.0]
Agile data science lifecycle is the most widely used framework, but only 25% of the survey participants state to follow a data science project methodology.
Professionals who adhere to a project methodology place greater emphasis on the project's potential risks and pitfalls.
arXiv Detail & Related papers (2022-01-17T09:50:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.