Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering

• URL: http://arxiv.org/abs/2211.03433v1
• Date: Mon, 7 Nov 2022 10:37:13 GMT
• Title: Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
• Abstract summary: We present a hybrid approach for dialogical data collection, which combines the intervention of human expert annotators over machine generated dialogues. The result of this work is DIALOCONAN, the first dataset comprising over 3000 fictitious multi-turn dialogues between a hater and an NGO operator, covering 6 targets of hate.
• Score: 15.905165019585942
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fighting online hate speech is a challenge that is usually addressed using Natural Language Processing via automatic detection and removal of hate content. Besides this approach, counter narratives have emerged as an effective tool employed by NGOs to respond to online hate on social media platforms. For this reason, Natural Language Generation is currently being studied as a way to automatize counter narrative writing. However, the existing resources necessary to train NLG models are limited to 2-turn interactions (a hate speech and a counter narrative as response), while in real life, interactions can consist of multiple turns. In this paper, we present a hybrid approach for dialogical data collection, which combines the intervention of human expert annotators over machine generated dialogues obtained using 19 different configurations. The result of this work is DIALOCONAN, the first dataset comprising over 3000 fictitious multi-turn dialogues between a hater and an NGO operator, covering 6 targets of hate.

Related papers

• Conversational Human Audio-visual Talking Dialogue Generation [50.95039085506777]
  We propose CHAT, a new dyadic interactive audio-visual dialogue generation framework.<n>Chat unifies large language models and talking face models with interactive audio and facial behaviour refinement modules.<n>Experiments show that CHAT outperforms existing related methods designed for similar tasks under both objective and subjective evaluations.
  arXiv  Detail & Related papers  (2026-07-02T22:17:55Z)
• CATCH-ME if you RAG: a dataset of Contextually Annotated multi-Turn Counterspeech against Hate and Misinformation Exchanges [9.965670989893267]
  We introduce the first large-scale, expert-curated, multilingual dataset of dialogues tackling the intersection of hate and misinformation.<n> Covering five languages and targeting hate directed at seven marginalized groups, this novel resource enables the training and evaluation of more persuasive, factually grounded counterspeech models.
  arXiv  Detail & Related papers  (2026-06-18T15:32:14Z)
• Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models [53.470209949659115]
  We propose a post-training alignment method that improves full spoken dialogue models through RL.<n>We address the four canonical axes of interactivity: handling turn-taking, back-channel pause, and user interruption.<n>An extra-based reward for response quality prevents semantic degradation.
  arXiv  Detail & Related papers  (2026-06-09T17:46:55Z)
• Advancing Hate Speech Detection with Transformers: Insights from the MetaHate [1.024113475677323]
  Social media platforms such as Twitter, Facebook, Instagram, Reddit, and others become breeding grounds for hate speech.<n>Deep learning approaches, such as vanilla recurrent neural networks (RNNs), long short-term memory (LSTM), and convolutional neural networks (CNNs) have achieved good results, but are often limited by issues such as long-term dependencies and inefficient parallelization.<n>This study represents the comprehensive exploration of transformer-based models for hate speech detection using the MetaHate dataset.
  arXiv  Detail & Related papers  (2025-08-06T22:36:17Z)
• REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation [51.97224538045096]
  We introduce REALTALK, a 21-day corpus of authentic messaging app dialogues. We compare EI attributes and persona consistency to understand the challenges posed by real-world dialogues. Our findings reveal that models struggle to simulate a user solely from dialogue history, while fine-tuning on specific user chats improves persona emulation.
  arXiv  Detail & Related papers  (2025-02-18T20:29:01Z)
• Self-Directed Turing Test for Large Language Models [56.64615470513102]
  The Turing test examines whether AIs can exhibit human-like behaviour in natural language conversations. Traditional Turing tests adopt a rigid dialogue format where each participant sends only one message each time. This paper proposes the Self-Directed Turing Test, which extends the original test with a burst dialogue format.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• Interactive Conversational Head Generation [68.76774230274076]
  We introduce a new conversation head generation benchmark for synthesizing behaviors of a single interlocutor in a face-to-face conversation. The capability to automatically synthesize interlocutors which can participate in long and multi-turn conversations is vital and offer benefits for various applications.
  arXiv  Detail & Related papers  (2023-07-05T08:06:26Z)
• CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network [52.85130555886915]
  CoSyn is a context-synergized neural network that explicitly incorporates user- and conversational context for detecting implicit hate speech in online conversations. We show that CoSyn outperforms all our baselines in detecting implicit hate speech with absolute improvements in the range of 1.24% - 57.8%.
  arXiv  Detail & Related papers  (2023-03-02T17:30:43Z)
• SalesBot: Transitioning from Chit-Chat to Task-Oriented Dialogues [22.89699254073016]
  How smoothly transitioning from social chatting to task-oriented dialogues is important for triggering business opportunities. This paper proposes a framework to automatically generate many dialogues without human involvement. The released data has a great potential of guiding future research directions and commercial activities.
  arXiv  Detail & Related papers  (2022-04-22T09:31:13Z)
• Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets [36.29939722039909]
  This paper describes the system proposed by team MIDAS-IIITD for HASOC 2021 subtask 2. It is one of the first shared tasks focusing on detecting hate speech from Hindi-English code-mixed conversations on Twitter. Our best performing system, a hard voting ensemble of Indic-BERT, XLM-RoBERTa, and Multilingual BERT, achieved a macro F1 score of 0.7253.
  arXiv  Detail & Related papers  (2021-12-18T19:27:33Z)
• Smoothing Dialogue States for Open Conversational Machine Reading [70.83783364292438]
  We propose an effective gating strategy by smoothing the two dialogue states in only one decoder and bridge decision making and question generation. Experiments on the OR-ShARC dataset show the effectiveness of our method, which achieves new state-of-the-art results.
  arXiv  Detail & Related papers  (2021-08-28T08:04:28Z)
• Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech [9.49544185939481]
  Off-the-shelf Natural Language Generation (NLG) methods are limited in that they generate commonplace, repetitive and safe responses. In this paper, we design a three- module pipeline approach to effectively improve the diversity and relevance. Our proposed pipeline first generates various counterspeech candidates by a generative model to promote diversity, then filters the ungrammatical ones using a BERT model, and finally selects the most relevant counterspeech response.
  arXiv  Detail & Related papers  (2021-06-03T06:54:03Z)
• CloneBot: Personalized Dialogue-Response Predictions [0.0]
  The project task was to create a model that, given a speaker ID, chat history, and an utterance query, can predict the response utterance in a conversation. The model is personalized for each speaker. This task can be a useful tool for building speech bots that talk in a human-like manner in a live conversation.
  arXiv  Detail & Related papers  (2021-03-31T01:15:37Z)
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots [62.295373408415365]
  We propose a personalized hybrid matching network (PHMN) for context-response matching. Our contributions are two-fold: 1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information. We evaluate our model on two large datasets with user identification, i.e., personalized dialogue Corpus Ubuntu (P- Ubuntu) and personalized Weibo dataset (P-Weibo)
  arXiv  Detail & Related papers  (2021-03-17T09:42:11Z)
• TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue [113.45485470103762]
  In this work, we unify nine human-human and multi-turn task-oriented dialogue datasets for language modeling. To better model dialogue behavior during pre-training, we incorporate user and system tokens into the masked language modeling.
  arXiv  Detail & Related papers  (2020-04-15T04:09:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.