ELF22: A Context-based Counter Trolling Dataset to Combat Internet Trolls

• URL: http://arxiv.org/abs/2208.00176v2
• Date: Tue, 2 Aug 2022 04:43:36 GMT
• Title: ELF22: A Context-based Counter Trolling Dataset to Combat Internet Trolls
• Authors: Huije Lee, Young Ju NA, Hoyun Song, Jisu Shin, Jong C. Park
• Abstract summary: We propose a novel dataset for automatic counter response generation. In particular, we constructed a pair-wise dataset that includes troll comments and counter responses with labeled response strategies. We demonstrate that the model fine-tuned on our dataset shows a significantly improved performance in strategy-controlled sentence generation.
• Score: 0.23624125155742054
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Online trolls increase social costs and cause psychological damage to individuals. With the proliferation of automated accounts making use of bots for trolling, it is difficult for targeted individual users to handle the situation both quantitatively and qualitatively. To address this issue, we focus on automating the method to counter trolls, as counter responses to combat trolls encourage community users to maintain ongoing discussion without compromising freedom of expression. For this purpose, we propose a novel dataset for automatic counter response generation. In particular, we constructed a pair-wise dataset that includes troll comments and counter responses with labeled response strategies, which enables models fine-tuned on our dataset to generate responses by varying counter responses according to the specified strategy. We conducted three tasks to assess the effectiveness of our dataset and evaluated the results through both automatic and human evaluation. In human evaluation, we demonstrate that the model fine-tuned on our dataset shows a significantly improved performance in strategy-controlled sentence generation.

Related papers

• Towards Effective Counter-Responses: Aligning Human Preferences with Strategies to Combat Online Trolling [9.598920004159696]
  This paper investigates whether humans have preferred strategies tailored to different types of trolling behaviors. We introduce a methodology for generating counter-responses to trolls by recommending appropriate RSs. The experimental results demonstrate that our proposed approach guides constructive discussion and reduces the negative effects of trolls.
  arXiv  Detail & Related papers  (2024-10-05T14:01:52Z)
• Self-Improving Customer Review Response Generation Based on LLMs [1.9274286238176854]
  SCRABLE represents an adaptive customer review response automation that enhances itself with self-optimizing prompts. We introduce an automatic scoring mechanism that mimics the role of a human evaluator to assess the quality of responses generated in customer review domains.
  arXiv  Detail & Related papers  (2024-05-06T20:50:17Z)
• Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting [74.68371461260946]
  SocialSense is a framework that induces a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics. Our method surpasses existing state-of-the-art in experimental evaluations for both zero-shot and supervised settings.
  arXiv  Detail & Related papers  (2023-10-20T06:17:02Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Measuring the Effect of Influential Messages on Varying Personas [67.1149173905004]
  We present a new task, Response Forecasting on Personas for News Media, to estimate the response a persona might have upon seeing a news message. The proposed task not only introduces personalization in the modeling but also predicts the sentiment polarity and intensity of each response. This enables more accurate and comprehensive inference on the mental state of the persona.
  arXiv  Detail & Related papers  (2023-05-25T21:01:00Z)
• Boosting Distress Support Dialogue Responses with Motivational Interviewing Strategy [4.264192013842096]
  We show how some response types could be rephrased into a more MI adherent form. We build several rephrasers by fine-tuning Blender and GPT3 to rephrase MI non-adherent "Advise without permission" responses into "Advise with permission"
  arXiv  Detail & Related papers  (2023-05-17T13:18:28Z)
• A Systematic Evaluation of Response Selection for Open Domain Dialogue [36.88551817451512]
  We curated a dataset where responses from multiple response generators produced for the same dialog context are manually annotated as appropriate (positive) and inappropriate (negative) We conduct a systematic evaluation of state-of-the-art methods for response selection, and demonstrate that both strategies of using multiple positive candidates and using manually verified hard negative candidates can bring in significant performance improvement in comparison to using the adversarial training data, e.g., increase of 3% and 13% in Recall@1 score, respectively.
  arXiv  Detail & Related papers  (2022-08-08T19:33:30Z)
• What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation [73.03318027164605]
  We propose to use information that can be automatically extracted from the next user utterance as a proxy to measure the quality of the previous system response. Our model generalizes across both spoken and written open-domain dialog corpora collected from real and paid users.
  arXiv  Detail & Related papers  (2022-03-25T22:09:52Z)
• Dialogue Response Ranking Training with Large-Scale Human Feedback Data [52.12342165926226]
  We leverage social media feedback data to build a large-scale training dataset for feedback prediction. We trained DialogRPT, a set of GPT-2 based models on 133M pairs of human feedback data. Our ranker outperforms the conventional dialog perplexity baseline with a large margin on predicting Reddit feedback.
  arXiv  Detail & Related papers  (2020-09-15T10:50:05Z)
• The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection [55.390442067381755]
  We show that grayscale data can be automatically constructed without human effort. Our method employs off-the-shelf response retrieval models and response generation models as automatic grayscale data generators. Experiments on three benchmark datasets and four state-of-the-art matching models show that the proposed approach brings significant and consistent performance improvements.
  arXiv  Detail & Related papers  (2020-04-06T06:34:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.