RAUCG: Retrieval-Augmented Unsupervised Counter Narrative Generation for
Hate Speech
- URL: http://arxiv.org/abs/2310.05650v1
- Date: Mon, 9 Oct 2023 12:01:26 GMT
- Title: RAUCG: Retrieval-Augmented Unsupervised Counter Narrative Generation for
Hate Speech
- Authors: Shuyu Jiang, Wenyi Tang, Xingshu Chen, Rui Tanga, Haizhou Wang and
Wenxian Wang
- Abstract summary: The Counter Narrative (CN) is a promising approach to combat online hate speech (HS) without infringing on freedom of speech.
Current automatic CN generation methods mainly rely on expert-authored datasets for training.
We propose Retrieval-Augmented Unsupervised Counter Narrative Generation (RAUCG) to automatically expand external counter-knowledge and map it into CNs.
- Score: 5.88043557914512
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Counter Narrative (CN) is a promising approach to combat online hate
speech (HS) without infringing on freedom of speech. In recent years, there has
been a growing interest in automatically generating CNs using natural language
generation techniques. However, current automatic CN generation methods mainly
rely on expert-authored datasets for training, which are time-consuming and
labor-intensive to acquire. Furthermore, these methods cannot directly obtain
and extend counter-knowledge from external statistics, facts, or examples. To
address these limitations, we propose Retrieval-Augmented Unsupervised Counter
Narrative Generation (RAUCG) to automatically expand external counter-knowledge
and map it into CNs in an unsupervised paradigm. Specifically, we first
introduce an SSF retrieval method to retrieve counter-knowledge from the
multiple perspectives of stance consistency, semantic overlap rate, and fitness
for HS. Then we design an energy-based decoding mechanism by quantizing
knowledge injection, countering and fluency constraints into differentiable
functions, to enable the model to build mappings from counter-knowledge to CNs
without expert-authored CN data. Lastly, we comprehensively evaluate model
performance in terms of language quality, toxicity, persuasiveness, relevance,
and success rate of countering HS, etc. Experimental results show that RAUCG
outperforms strong baselines on all metrics and exhibits stronger
generalization capabilities, achieving significant improvements of +2.0% in
relevance and +4.5% in success rate of countering metrics. Moreover, RAUCG
enabled GPT2 to outperform T0 in all metrics, despite the latter being
approximately eight times larger than the former. Warning: This paper may
contain offensive or upsetting content!
Related papers
- A LLM-Based Ranking Method for the Evaluation of Automatic Counter-Narrative Generation [14.064465097974836]
This paper proposes a novel approach to asses generated Counter Narratives (CNs) that consists on the use of a Large Language Model (LLM) as a evaluator.
By comparing generated CNs pairwise in a tournament-style format, we establish a model ranking pipeline that achieves a correlation of $0.88$ with human preference.
We conclude that chat-aligned models in ZS are the best option for carrying out the task, provided they do not refuse to generate an answer due to security concerns.
arXiv Detail & Related papers (2024-06-21T15:11:33Z) - Improving the Robustness of Knowledge-Grounded Dialogue via Contrastive
Learning [71.8876256714229]
We propose an entity-based contrastive learning framework for improving the robustness of knowledge-grounded dialogue systems.
Our method achieves new state-of-the-art performance in terms of automatic evaluation scores.
arXiv Detail & Related papers (2024-01-09T05:16:52Z) - Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language
Models [57.80297731039275]
Chain-of-Noting (CoN) is a novel approach aimed at improving the robustness of RALMs in facing noisy, irrelevant documents and in handling unknown scenarios.
CoN achieves an average improvement of +7.9 in EM score given entirely noisy retrieved documents and +10.5 in rejection rates for real-time questions that fall outside the pre-training knowledge scope.
arXiv Detail & Related papers (2023-11-15T18:54:53Z) - CTC-based Non-autoregressive Speech Translation [51.37920141751813]
We investigate the potential of connectionist temporal classification for non-autoregressive speech translation.
We develop a model consisting of two encoders that are guided by CTC to predict the source and target texts.
Experiments on the MuST-C benchmarks show that our NAST model achieves an average BLEU score of 29.5 with a speed-up of 5.67$times$.
arXiv Detail & Related papers (2023-05-27T03:54:09Z) - Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense
Reasoning [85.1541170468617]
This paper reconsiders the nature of commonsense reasoning and proposes a novel commonsense reasoning metric, Non-Replacement Confidence (NRC)
Our proposed novel method boosts zero-shot performance on two commonsense reasoning benchmark datasets and further seven commonsense question-answering datasets.
arXiv Detail & Related papers (2022-08-23T14:42:14Z) - Using Pre-Trained Language Models for Producing Counter Narratives
Against Hate Speech: a Comparative Study [17.338923885534193]
We present an extensive study on the use of pre-trained language models for the task of automatic Counter Narrative (CN) generation.
We first present a comparative study to determine whether there is a particular Language Model (or class of LMs) and a particular decoding mechanism that are the most appropriate to generate CNs.
Findings show that autoregressive models combined with decodings are the most promising.
arXiv Detail & Related papers (2022-04-04T12:44:47Z) - Graph Backdoor [53.70971502299977]
We present GTA, the first backdoor attack on graph neural networks (GNNs)
GTA departs in significant ways: it defines triggers as specific subgraphs, including both topological structures and descriptive features.
It can be instantiated for both transductive (e.g., node classification) and inductive (e.g., graph classification) tasks.
arXiv Detail & Related papers (2020-06-21T19:45:30Z) - Control, Generate, Augment: A Scalable Framework for Multi-Attribute
Text Generation [22.70189685469752]
We introduce CGA, a conditional VAE architecture, to control, generate, and augment text.
We show the value of the individual model components in an ablation study.
We show high quality, diversity and attribute control in the generated sentences through a series of automatic and human assessments.
arXiv Detail & Related papers (2020-04-30T17:31:16Z) - Characterizing Speech Adversarial Examples Using Self-Attention U-Net
Enhancement [102.48582597586233]
We present a U-Net based attention model, U-Net$_At$, to enhance adversarial speech signals.
We conduct experiments on the automatic speech recognition (ASR) task with adversarial audio attacks.
arXiv Detail & Related papers (2020-03-31T02:16:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.