Measuring the Mental Health of Content Reviewers, a Systematic Review

• URL: http://arxiv.org/abs/2502.00244v1
• Date: Sat, 01 Feb 2025 00:50:15 GMT
• Title: Measuring the Mental Health of Content Reviewers, a Systematic Review
• Authors: Alexandra Gonzalez, J. Nathan Matias,
• Abstract summary: Many workers report long-term, potentially irreversible psychological harm. This work is similar to activities that cause psychological harm to other kinds of helping professionals even after small doses of exposure. This systematic review summarizes psychological measures from other professions and relates them to the experiences of content reviewers.
• Score: 50.06646946044604
• License:
• Abstract: Artificial intelligence and social computing rely on hundreds of thousands of content reviewers to classify high volumes of harmful and forbidden content. Many workers report long-term, potentially irreversible psychological harm. This work is similar to activities that cause psychological harm to other kinds of helping professionals even after small doses of exposure. Yet researchers struggle to measure the mental health of content reviewers well enough to inform diagnoses, evaluate workplace improvements, hold employers accountable, or advance scientific understanding. This systematic review summarizes psychological measures from other professions and relates them to the experiences of content reviewers. After identifying 1,673 potential papers, we reviewed 143 that validate measures in related occupations. We summarize the uses of psychological measurement for content reviewing, differences between clinical and research measures, and 12 measures that are adaptable to content reviewing. We find serious gaps in measurement validity in regions where content review labor is common. Overall, we argue for reliable measures of content reviewer mental health that match the nature of the work and are culturally-relevant.

Related papers

• Sentiment-guided Commonsense-aware Response Generation for Mental Health Counseling [28.17719749654601]
  We propose EmpRes, a novel sentiment-guided mechanism incorporating commonsense awareness for generating responses. By leveraging foundation models and harnessing commonsense knowledge, EmpRes aims to generate responses that effectively shape the client's sentiment towards positivity. Our findings show that 91% of users find the system effective, 80% express satisfaction, and over 85.45% convey a willingness to continue using the interface and recommend it to others.
  arXiv  Detail & Related papers  (2025-01-06T15:41:52Z)
• PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents [68.50571379012621]
  Psychological measurement is essential for mental health, self-understanding, and personal development. PsychoGAT (Psychological Game AgenTs) achieves statistically significant excellence in psychometric metrics such as reliability, convergent validity, and discriminant validity.
  arXiv  Detail & Related papers  (2024-02-19T18:00:30Z)
• An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives [7.564560899044939]
  We conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques.
  arXiv  Detail & Related papers  (2023-10-25T21:37:57Z)
• Handwriting and Drawing for Depression Detection: A Preliminary Study [53.11777541341063]
  Short-term covid effects on mental health were a significant increase in anxiety and depressive symptoms. The aim of this study is to use a new tool, the online handwriting and drawing analysis, to discriminate between healthy individuals and depressed patients.
  arXiv  Detail & Related papers  (2023-02-05T22:33:49Z)
• Multi-task Learning for Personal Health Mention Detection on Social Media [70.23889100356091]
  This research employs a multitask learning framework to leverage available annotated data to improve the performance on the main task. We focus on incorporating emotional information into our target task by using emotion detection as an auxiliary task.
  arXiv  Detail & Related papers  (2022-12-09T23:49:00Z)
• Semantic Similarity Models for Depression Severity Estimation [53.72188878602294]
  This paper presents an efficient semantic pipeline to study depression severity in individuals based on their social media writings. We use test user sentences for producing semantic rankings over an index of representative training sentences corresponding to depressive symptoms and severity levels. We evaluate our methods on two Reddit-based benchmarks, achieving 30% improvement over state of the art in terms of measuring depression severity.
  arXiv  Detail & Related papers  (2022-11-14T18:47:26Z)
• Interpretability by design using computer vision for behavioral sensing in child and adolescent psychiatry [3.975358343371988]
  We use machine learning to derive behavioral codes or concepts of a gold standard behavioral rating system. Our ratings were comparable to human expert ratings for negative emotions, activity-level/arousal and anxiety.
  arXiv  Detail & Related papers  (2022-07-11T09:07:08Z)
• Social Behavior and Mental Health: A Snapshot Survey under COVID-19 Pandemic [6.5721468981020665]
  COVID-19 pandemic has changed the way we live, study, socialize and recreate. There are growing researches that leverage online social media analysis to detect and assess user's mental status.
  arXiv  Detail & Related papers  (2021-05-17T21:08:03Z)
• Assessing the Severity of Health States based on Social Media Posts [62.52087340582502]
  We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.
  arXiv  Detail & Related papers  (2020-09-21T03:45:14Z)
• Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines [19.40714760075466]
  We provide an introduction to psychometric theory for the evaluation of measurement instruments for software engineering researchers. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology.
  arXiv  Detail & Related papers  (2020-05-20T11:03:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.