Opinions are Made to be Changed: Temporally Adaptive Stance Classification

• URL: http://arxiv.org/abs/2108.12476v1
• Date: Fri, 27 Aug 2021 19:47:31 GMT
• Title: Opinions are Made to be Changed: Temporally Adaptive Stance Classification
• Authors: Rabab Alkhalifa, Elena Kochkina, Arkaitz Zubiaga
• Abstract summary: We introduce two novel large-scale, longitudinal stance datasets. We evaluate the performance persistence of stance classifiers over time and demonstrate how it decays as the temporal gap between training and testing data increases. We propose and compare several approaches to embedding adaptation and find that the Incremental Temporal Alignment (ITA) model leads to the best results in reducing performance drop over time.
• Score: 9.061088449712859
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given the rapidly evolving nature of social media and people's views, word usage changes over time. Consequently, the performance of a classifier trained on old textual data can drop dramatically when tested on newer data. While research in stance classification has advanced in recent years, no effort has been invested in making these classifiers have persistent performance over time. To study this phenomenon we introduce two novel large-scale, longitudinal stance datasets. We then evaluate the performance persistence of stance classifiers over time and demonstrate how it decays as the temporal gap between training and testing data increases. We propose a novel approach to mitigate this performance drop, which is based on temporal adaptation of the word embeddings used for training the stance classifier. This enables us to make use of readily available unlabelled data from the current time period instead of expensive annotation efforts. We propose and compare several approaches to embedding adaptation and find that the Incremental Temporal Alignment (ITA) model leads to the best results in reducing performance drop over time.

Related papers

• BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models [20.88680592729709]
  We propose a novel backpropagation-free algorithm BaFTA for test-time adaptation of vision-language models. BaFTA directly estimates class centroids using online clustering within a projected embedding space. We demonstrate that BaFTA consistently outperforms state-of-the-art test-time adaptation methods in both effectiveness and efficiency.
  arXiv  Detail & Related papers  (2024-06-17T08:16:24Z)
• Improving Sampling Methods for Fine-tuning SentenceBERT in Text Streams [49.3179290313959]
  This study explores the efficacy of seven text sampling methods designed to selectively fine-tune language models. We precisely assess the impact of these methods on fine-tuning the SBERT model using four different loss functions. Our findings indicate that Softmax loss and Batch All Triplets loss are particularly effective for text stream classification.
  arXiv  Detail & Related papers  (2024-03-18T23:41:52Z)
• Keeping in Time: Adding Temporal Context to Sentiment Analysis Models [0.0]
  This paper presents a state-of-the-art solution to the LongEval CLEF 2023 Lab Task 2: LongEval-Classification. The goal of this task is to improve and preserve the performance of sentiment analysis models across shorter and longer time periods.
  arXiv  Detail & Related papers  (2023-09-24T06:38:21Z)
• Time Series Contrastive Learning with Information-Aware Augmentations [57.45139904366001]
  A key component of contrastive learning is to select appropriate augmentations imposing some priors to construct feasible positive samples. How to find the desired augmentations of time series data that are meaningful for given contrastive learning tasks and datasets remains an open question. We propose a new contrastive learning approach with information-aware augmentations, InfoTS, that adaptively selects optimal augmentations for time series representation learning.
  arXiv  Detail & Related papers  (2023-03-21T15:02:50Z)
• Building for Tomorrow: Assessing the Temporal Persistence of Text Classifiers [18.367109894193486]
  Performance of text classification models can drop over time when new data to be classified is more distant in time from the data used for training. This raises important research questions on the design of text classification models intended to persist over time. We perform longitudinal classification experiments on three datasets spanning between 6 and 19 years.
  arXiv  Detail & Related papers  (2022-05-11T12:21:14Z)
• A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
  Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
  arXiv  Detail & Related papers  (2022-03-10T08:58:18Z)
• Temporal Effects on Pre-trained Models for Language Processing Tasks [9.819970078135343]
  We present a set of experiments with systems powered by large neural pretrained representations for English to demonstrate that em temporal model deterioration is not as big a concern. It is however the case that em temporal domain adaptation is beneficial, with better performance for a given time period possible when the system is trained on temporally more recent data.
  arXiv  Detail & Related papers  (2021-11-24T20:44:12Z)
• Semi-supervised Long-tailed Recognition using Alternate Sampling [95.93760490301395]
  Main challenges in long-tailed recognition come from the imbalanced data distribution and sample scarcity in its tail classes. We propose a new recognition setting, namely semi-supervised long-tailed recognition. We demonstrate significant accuracy improvements over other competitive methods on two datasets.
  arXiv  Detail & Related papers  (2021-05-01T00:43:38Z)
• A Closer Look at Temporal Sentence Grounding in Videos: Datasets and Metrics [70.45937234489044]
  We re- organize two widely-used TSGV datasets (Charades-STA and ActivityNet Captions) to make it different from the training split. We introduce a new evaluation metric "dR@$n$,IoU@$m$" to calibrate the basic IoU scores. All the results demonstrate that the re-organized datasets and new metric can better monitor the progress in TSGV.
  arXiv  Detail & Related papers  (2021-01-22T09:59:30Z)
• On the Exploration of Incremental Learning for Fine-grained Image Retrieval [45.48333682748607]
  We consider the problem of fine-grained image retrieval in an incremental setting, when new categories are added over time. We propose an incremental learning method to mitigate retrieval performance degradation caused by the forgetting issue. Our method effectively mitigates the catastrophic forgetting on the original classes while achieving high performance on the new classes.
  arXiv  Detail & Related papers  (2020-10-15T21:07:44Z)
• Active Learning for Coreference Resolution using Discrete Annotation [76.36423696634584]
  We improve upon pairwise annotation for active learning in coreference resolution. We ask annotators to identify mention antecedents if a presented mention pair is deemed not coreferent. In experiments with existing benchmark coreference datasets, we show that the signal from this additional question leads to significant performance gains per human-annotation hour.
  arXiv  Detail & Related papers  (2020-04-28T17:17:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.