Combating Temporal Drift in Crisis with Adapted Embeddings
- URL: http://arxiv.org/abs/2104.08535v1
- Date: Sat, 17 Apr 2021 13:11:41 GMT
- Title: Combating Temporal Drift in Crisis with Adapted Embeddings
- Authors: Kevin Stowe, Iryna Gurevych
- Abstract summary: Language usage changes over time, and this can impact the effectiveness of NLP systems.
This work investigates methods for adapting to changing discourse during crisis events.
- Score: 58.4558720264897
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Language usage changes over time, and this can impact the effectiveness of
NLP systems. This work investigates methods for adapting to changing discourse
during crisis events. We explore social media data during crisis, for which
effective, time-sensitive methods are necessary. We experiment with two
separate methods to accommodate changing data: temporal pretraining, which uses
unlabeled data for the target time periods to train better language models, and
a model of embedding shift based on tools for analyzing semantic change. This
shift allows us to counteract temporal drift by normalizing incoming data based
on observed patterns of language change. Simulating scenarios in which we lack
access to incoming labeled data, we demonstrate the effectiveness of these
methods for a wide variety of crises, showing we can improve performance by up
to 8.0 F1 score for relevance classification across datasets.
Related papers
- Using a Local Surrogate Model to Interpret Temporal Shifts in Global Annual Data [5.669106489320257]
This paper focuses on explaining changes over time in globally-sourced, annual temporal data.
We employ Local Interpretable Model-agnostic Explanations (LIME) to shed light on national happiness indices, economic freedom, and population metrics.
arXiv Detail & Related papers (2024-04-18T03:17:45Z) - Improving Sampling Methods for Fine-tuning SentenceBERT in Text Streams [49.3179290313959]
This study explores the efficacy of seven text sampling methods designed to selectively fine-tune language models.
We precisely assess the impact of these methods on fine-tuning the SBERT model using four different loss functions.
Our findings indicate that Softmax loss and Batch All Triplets loss are particularly effective for text stream classification.
arXiv Detail & Related papers (2024-03-18T23:41:52Z) - FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing [60.629222280633606]
We investigate scheduled unfreezing algorithms for fine-tuning task adapters.
Experiments show scheduled unfreezing methods close the gap to full fine-tuning and achieve stronger cross-lingual transfer performance.
We propose a general scheduled unfreezing algorithm that achieves an average of 2 points improvement over four datasets.
arXiv Detail & Related papers (2023-01-13T11:26:53Z) - Improving Temporal Generalization of Pre-trained Language Models with
Lexical Semantic Change [28.106524698188675]
Recent research has revealed that neural language models at scale suffer from poor temporal generalization capability.
We propose a simple yet effective lexical-level masking strategy to post-train a converged language model.
arXiv Detail & Related papers (2022-10-31T08:12:41Z) - Time-Varying Propensity Score to Bridge the Gap between the Past and Present [104.46387765330142]
We introduce a time-varying propensity score that can detect gradual shifts in the distribution of data.
We demonstrate different ways of implementing it and evaluate it on a variety of problems.
arXiv Detail & Related papers (2022-10-04T07:21:49Z) - Improving Classifier Training Efficiency for Automatic Cyberbullying
Detection with Feature Density [58.64907136562178]
We study the effectiveness of Feature Density (FD) using different linguistically-backed feature preprocessing methods.
We hypothesise that estimating dataset complexity allows for the reduction of the number of required experiments.
The difference in linguistic complexity of datasets allows us to additionally discuss the efficacy of linguistically-backed word preprocessing.
arXiv Detail & Related papers (2021-11-02T15:48:28Z) - Learning Neural Models for Natural Language Processing in the Face of
Distributional Shift [10.990447273771592]
The dominating NLP paradigm of training a strong neural predictor to perform one task on a specific dataset has led to state-of-the-art performance in a variety of applications.
It builds upon the assumption that the data distribution is stationary, ie. that the data is sampled from a fixed distribution both at training and test time.
This way of training is inconsistent with how we as humans are able to learn from and operate within a constantly changing stream of information.
It is ill-adapted to real-world use cases where the data distribution is expected to shift over the course of a model's lifetime
arXiv Detail & Related papers (2021-09-03T14:29:20Z) - Overcoming Conflicting Data when Updating a Neural Semantic Parser [5.471925005642665]
We show how to use a small amount of new data to update a task-oriented semantic parsing model when the desired output for some examples has changed.
When making updates in this way, one potential problem that arises is the presence of conflicting data.
We show that the presence of conflicting data greatly hinders learning of an update, then explore several methods to mitigate its effect.
arXiv Detail & Related papers (2020-10-23T21:19:03Z) - Change Point Detection in Time Series Data using Autoencoders with a
Time-Invariant Representation [69.34035527763916]
Change point detection (CPD) aims to locate abrupt property changes in time series data.
Recent CPD methods demonstrated the potential of using deep learning techniques, but often lack the ability to identify more subtle changes in the autocorrelation statistics of the signal.
We employ an autoencoder-based methodology with a novel loss function, through which the used autoencoders learn a partially time-invariant representation that is tailored for CPD.
arXiv Detail & Related papers (2020-08-21T15:03:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.