How WEIRD is Usable Privacy and Security Research? (Extended Version)
- URL: http://arxiv.org/abs/2305.05004v2
- Date: Sun, 8 Oct 2023 22:22:22 GMT
- Title: How WEIRD is Usable Privacy and Security Research? (Extended Version)
- Authors: Ayako A. Hasegawa, Daisuke Inoue, Mitsuaki Akiyama
- Abstract summary: We conducted a literature review to understand the extent to which participant samples in UPS papers were from WEIRD countries.
Geographic and linguistic barriers in the study methods and recruitment methods may cause researchers to conduct user studies locally.
- Score: 7.669758543344074
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In human factor fields such as human-computer interaction (HCI) and
psychology, researchers have been concerned that participants mostly come from
WEIRD (Western, Educated, Industrialized, Rich, and Democratic) countries. This
WEIRD skew may hinder understanding of diverse populations and their cultural
differences. The usable privacy and security (UPS) field has inherited many
research methodologies from research on human factor fields. We conducted a
literature review to understand the extent to which participant samples in UPS
papers were from WEIRD countries and the characteristics of the methodologies
and research topics in each user study recruiting Western or non-Western
participants. We found that the skew toward WEIRD countries in UPS is greater
than that in HCI. Geographic and linguistic barriers in the study methods and
recruitment methods may cause researchers to conduct user studies locally. In
addition, many papers did not report participant demographics, which could
hinder the replication of the reported studies, leading to low reproducibility.
To improve geographic diversity, we provide the suggestions including
facilitate replication studies, address geographic and linguistic issues of
study/recruitment methods, and facilitate research on the topics for non-WEIRD
populations.
Related papers
- A Survey Forest Diagram : Gain a Divergent Insight View on a Specific Research Topic [2.699900017799093]
The use of Generative AI for information retrieval and question-answering has become popular for conducting research surveys.
This study aims to develop an in-depth Survey Forest Diagram that guides novice researchers in divergent thinking about the research topic.
arXiv Detail & Related papers (2024-07-24T08:17:37Z) - Who's actually being Studied? A Call for Population Analysis in Software Engineering Research [0.0]
Population analysis is crucial for ensuring that empirical software engineering (ESE) research is representative and its findings are valid.
We explore the challenges ranging from analysing populations of individual software engineers to organizations and projects.
arXiv Detail & Related papers (2024-04-23T14:48:11Z) - ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models [56.08917291606421]
ResearchAgent is a large language model-powered research idea writing agent.
It generates problems, methods, and experiment designs while iteratively refining them based on scientific literature.
We experimentally validate our ResearchAgent on scientific publications across multiple disciplines.
arXiv Detail & Related papers (2024-04-11T13:36:29Z) - SurveyAgent: A Conversational System for Personalized and Efficient Research Survey [50.04283471107001]
This paper introduces SurveyAgent, a novel conversational system designed to provide personalized and efficient research survey assistance to researchers.
SurveyAgent integrates three key modules: Knowledge Management for organizing papers, Recommendation for discovering relevant literature, and Query Answering for engaging with content on a deeper level.
Our evaluation demonstrates SurveyAgent's effectiveness in streamlining research activities, showcasing its capability to facilitate how researchers interact with scientific literature.
arXiv Detail & Related papers (2024-04-09T15:01:51Z) - The State of Pilot Study Reporting in Crowdsourcing: A Reflection on
Best Practices and Guidelines [12.782288692460565]
A lack of details surrounding pilot studies in crowdsourcing research hinders the replication of studies and the reproduction of findings.
We conducted a systematic literature review on the current state of pilot study reporting at the intersection of crowdsourcing and HCI research.
We formulate a set of best practice guidelines for reporting crowd pilot studies in crowdsourcing research.
arXiv Detail & Related papers (2023-12-13T12:13:40Z) - A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and
Why? [84.46288849132634]
We propose a systematic framework for analyzing the evolution of research topics in a scientific field using causal discovery and inference techniques.
We define three variables to encompass diverse facets of the evolution of research topics within NLP.
We utilize a causal discovery algorithm to unveil the causal connections among these variables using observational data.
arXiv Detail & Related papers (2023-05-22T11:08:00Z) - WEIRD FAccTs: How Western, Educated, Industrialized, Rich, and
Democratic is FAccT? [8.12219922021227]
Studies conducted on Western, Educated, Industrialized, Rich, and Democratic (WEIRD) samples are considered atypical of the world's population.
This study aims to quantify the extent to which the ACM FAccT conference relies on WEIRD samples.
arXiv Detail & Related papers (2023-05-10T18:52:09Z) - Language Generation Models Can Cause Harm: So What Can We Do About It?
An Actionable Survey [50.58063811745676]
This work provides a survey of practical methods for addressing potential threats and societal harms from language generation models.
We draw on several prior works' of language model risks to present a structured overview of strategies for detecting and ameliorating different kinds of risks/harms of language generators.
arXiv Detail & Related papers (2022-10-14T10:43:39Z) - Award rate inequities in biomedical research [55.850540873687386]
The authors performed an analysis of 14,263 biomedical research proposals with proposed start dates between 2010-2022 from the University of Michigan Medical School.
There is a clear relationship between race/ethnicity and rates of proposal award.
Black/African American and Asian researchers appear disadvantaged across all submission categories relative to White researchers.
arXiv Detail & Related papers (2022-06-14T14:05:39Z) - Human Factors in Security Research: Lessons Learned from 2008-2018 [8.255966566768484]
We focus our analysis on the research on the crucial population of experts, whose human errors can impact many systems at once.
We analyzed the past decade of human factors research in security and privacy, identifying 557 relevant publications.
arXiv Detail & Related papers (2021-03-24T15:58:05Z) - Learnings from Frontier Development Lab and SpaceML -- AI Accelerators
for NASA and ESA [57.06643156253045]
Research with AI and ML technologies lives in a variety of settings with often asynchronous goals and timelines.
We perform a case study of the Frontier Development Lab (FDL), an AI accelerator under a public-private partnership from NASA and ESA.
FDL research follows principled practices that are grounded in responsible development, conduct, and dissemination of AI research.
arXiv Detail & Related papers (2020-11-09T21:23:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.