Into the crossfire: evaluating the use of a language model to crowdsource gun violence reports

• URL: http://arxiv.org/abs/2401.12989v1
• Date: Tue, 16 Jan 2024 14:40:54 GMT
• Title: Into the crossfire: evaluating the use of a language model to crowdsource gun violence reports
• Authors: Adriano Belisario, Scott Hale, Luc Rocher
• Abstract summary: We propose a fine-tuned BERT-based model trained on Twitter texts to distinguish gun violence reports from ordinary Portuguese texts. We study and interview Brazilian analysts who continuously fact-check social media texts to identify new gun violence events.
• Score: 0.21485350418225244
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Gun violence is a pressing and growing human rights issue that affects nearly every dimension of the social fabric, from healthcare and education to psychology and the economy. Reliable data on firearm events is paramount to developing more effective public policy and emergency responses. However, the lack of comprehensive databases and the risks of in-person surveys prevent human rights organizations from collecting needed data in most countries. Here, we partner with a Brazilian human rights organization to conduct a systematic evaluation of language models to assist with monitoring real-world firearm events from social media data. We propose a fine-tuned BERT-based model trained on Twitter (now X) texts to distinguish gun violence reports from ordinary Portuguese texts. Our model achieves a high AUC score of 0.97. We then incorporate our model into a web application and test it in a live intervention. We study and interview Brazilian analysts who continuously fact-check social media texts to identify new gun violence events. Qualitative assessments show that our solution helped all analysts use their time more efficiently and expanded their search capacities. Quantitative assessments show that the use of our model was associated with more analysts' interactions with online users reporting gun violence. Taken together, our findings suggest that modern Natural Language Processing techniques can help support the work of human rights organizations.

Related papers

• Mapping Violence: Developing an Extensive Framework to Build a Bangla Sectarian Expression Dataset from Social Media Interactions [1.2618555186247333]
  We have developed the first comprehensive framework for the automatic detection of communal violence markers in online Bangla content. Our workflow introduces a 7-step expert annotation process incorporating insights from social scientists, linguists, and psychologists. By presenting data statistics and benchmarking performance using this dataset, we have determined that, aside from the category of Non-communal violence, Religio-communal violence is particularly pervasive in Bangla text.
  arXiv  Detail & Related papers  (2024-04-17T21:09:13Z)
• A Call to Arms: AI Should be Critical for Social Media Analysis of Conflict Zones [5.479613761646247]
  This paper presents preliminary, transdisciplinary work using computer vision to identify specific weapon systems and the insignias of the armed groups using them. There is potential to not only track how weapons are distributed through networks of armed units but also to track which types of weapons are being used by the different types of state and non-state military actors in Ukraine. Such a system could ultimately be used to understand conflicts in real-time, including where humanitarian and medical aid is most needed.
  arXiv  Detail & Related papers  (2023-11-01T19:49:32Z)
• Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting [74.68371461260946]
  SocialSense is a framework that induces a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics. Our method surpasses existing state-of-the-art in experimental evaluations for both zero-shot and supervised settings.
  arXiv  Detail & Related papers  (2023-10-20T06:17:02Z)
• Measuring the Effect of Influential Messages on Varying Personas [67.1149173905004]
  We present a new task, Response Forecasting on Personas for News Media, to estimate the response a persona might have upon seeing a news message. The proposed task not only introduces personalization in the modeling but also predicts the sentiment polarity and intensity of each response. This enables more accurate and comprehensive inference on the mental state of the persona.
  arXiv  Detail & Related papers  (2023-05-25T21:01:00Z)
• Understanding Lexical Biases when Identifying Gang-related Social Media Communications [18.301221486244263]
  We use a binary logistic classifier to identify gang-related tweets in Chicago. We find that the language of a tweet is highly relevant and that uses of big data'' methods or machine learning models need to better understand how language impacts the model's performance.
  arXiv  Detail & Related papers  (2023-04-22T21:51:49Z)
• PAL: Persona-Augmented Emotional Support Conversation Generation [54.069321178816686]
  Due to the lack of human resources for mental health support, there is an increasing demand for employing conversational agents for support. Recent work has demonstrated the effectiveness of dialogue models in providing emotional support. We propose a framework for dynamically inferring and modeling seekers' persona.
  arXiv  Detail & Related papers  (2022-12-19T04:12:54Z)
• Modeling Political Activism around Gun Debate via Social Media [16.571752603108344]
  We employ social media signals to examine the predictors of offline political activism, at both population and individual level. We show that it is possible to classify the stance of users on the gun issue, especially when network information is available.
  arXiv  Detail & Related papers  (2022-04-30T17:17:08Z)
• LaMDA: Language Models for Dialog Applications [75.75051929981933]
  LaMDA is a family of Transformer-based neural language models specialized for dialog. Fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements.
  arXiv  Detail & Related papers  (2022-01-20T15:44:37Z)
• FacTeR-Check: Semi-automated fact-checking through Semantic Similarity and Natural Language Inference [61.068947982746224]
  FacTeR-Check enables retrieving fact-checked information, unchecked claims verification and tracking dangerous information over social media. The architecture is validated using a new dataset called NLI19-SP that is publicly released with COVID-19 related hoaxes and tweets from Spanish social media. Our results show state-of-the-art performance on the individual benchmarks, as well as producing useful analysis of the evolution over time of 61 different hoaxes.
  arXiv  Detail & Related papers  (2021-10-27T15:44:54Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.