Dynamic Forecasting of Conversation Derailment
- URL: http://arxiv.org/abs/2110.05111v1
- Date: Mon, 11 Oct 2021 09:33:34 GMT
- Title: Dynamic Forecasting of Conversation Derailment
- Authors: Yova Kementchedjhieva and Anders S{\o}gaard
- Abstract summary: We apply a pretrained language encoder to the task, which outperforms earlier approaches.
We experiment with shifting the training paradigm for the task from a static to a dynamic one to increase the forecast horizon.
This approach shows mixed results: in a high-quality data setting, a longer average forecast horizon can be achieved at the cost of a small drop in F1; in a low-quality data setting, dynamic training propagates the noise and is highly detrimental to performance.
- Score: 8.62483598990205
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Online conversations can sometimes take a turn for the worse, either due to
systematic cultural differences, accidental misunderstandings, or mere malice.
Automatically forecasting derailment in public online conversations provides an
opportunity to take early action to moderate it. Previous work in this space is
limited, and we extend it in several ways. We apply a pretrained language
encoder to the task, which outperforms earlier approaches. We further
experiment with shifting the training paradigm for the task from a static to a
dynamic one to increase the forecast horizon. This approach shows mixed
results: in a high-quality data setting, a longer average forecast horizon can
be achieved at the cost of a small drop in F1; in a low-quality data setting,
however, dynamic training propagates the noise and is highly detrimental to
performance.
Related papers
- TrACT: A Training Dynamics Aware Contrastive Learning Framework for Long-tail Trajectory Prediction [7.3292387742640415]
We propose to incorporate richer training dynamics information into a prototypical contrastive learning framework.
We conduct empirical evaluations of our approach using two large-scale naturalistic datasets.
arXiv Detail & Related papers (2024-04-18T23:12:46Z) - Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline
Pre-Training with Model Based Augmentation [59.899714450049494]
offline pre-training can produce sub-optimal policies and lead to degraded online reinforcement learning performance.
We propose a model-based data augmentation strategy to maximize the benefits of offline reinforcement learning pre-training and reduce the scale of data needed to be effective.
arXiv Detail & Related papers (2023-12-15T14:49:41Z) - Hashing it Out: Predicting Unhealthy Conversations on Twitter [0.17175853976270528]
We show that an Attention-based BERT architecture, pre-trained on a large Twitter corpus, is efficient and effective in making such predictions.
This work lays the foundation for a practical tool to encourage better interactions on one of the most ubiquitous social media platforms.
arXiv Detail & Related papers (2023-11-17T15:49:11Z) - Understanding and Mitigating the Label Noise in Pre-training on
Downstream Tasks [91.15120211190519]
This paper aims to understand the nature of noise in pre-training datasets and to mitigate its impact on downstream tasks.
We propose a light-weight black-box tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise.
arXiv Detail & Related papers (2023-09-29T06:18:15Z) - Conversation Modeling to Predict Derailment [15.45515784064555]
The ability to predict whether ongoing conversations are likely to derail could provide valuable real-time insight to interlocutors and moderators.
Some works attempt to make dynamic prediction as the conversation develops, but fail to incorporate multisource information, such as conversation structure and distance to derailment.
We propose a hierarchical transformer-based framework that combines utterance-level and conversation-level information to capture fine-grained contextual semantics.
arXiv Detail & Related papers (2023-03-20T15:10:45Z) - SPIRAL: Self-supervised Perturbation-Invariant Representation Learning
for Speech Pre-Training [25.80559992732508]
SPIRAL works by learning denoising representation of perturbed data in a teacher-student framework.
We address the problem of noise-robustness that is critical to real-world speech applications.
arXiv Detail & Related papers (2022-01-25T09:53:36Z) - Mixing between the Cross Entropy and the Expectation Loss Terms [89.30385901335323]
Cross entropy loss tends to focus on hard to classify samples during training.
We show that adding to the optimization goal the expectation loss helps the network to achieve better accuracy.
Our experiments show that the new training protocol improves performance across a diverse set of classification domains.
arXiv Detail & Related papers (2021-09-12T23:14:06Z) - Exploiting Unsupervised Data for Emotion Recognition in Conversations [76.01690906995286]
Emotion Recognition in Conversations (ERC) aims to predict the emotional state of speakers in conversations.
The available supervised data for the ERC task is limited.
We propose a novel approach to leverage unsupervised conversation data.
arXiv Detail & Related papers (2020-10-02T13:28:47Z) - Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models
via Continual Learning [74.25168207651376]
Fine-tuning pre-trained language models to downstream cross-lingual tasks has shown promising results.
We leverage continual learning to preserve the cross-lingual ability of the pre-trained model when we fine-tune it to downstream tasks.
Our methods achieve better performance than other fine-tuning baselines on the zero-shot cross-lingual part-of-speech tagging and named entity recognition tasks.
arXiv Detail & Related papers (2020-04-29T14:07:18Z) - Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less
Forgetting [66.45372974713189]
We propose a recall and learn mechanism, which adopts the idea of multi-task learning and jointly learns pretraining tasks and downstream tasks.
Experiments show that our method achieves state-of-the-art performance on the GLUE benchmark.
We provide open-source RecAdam, which integrates the proposed mechanisms into Adam to facility the NLP community.
arXiv Detail & Related papers (2020-04-27T08:59:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.