Information Type Classification with Contrastive Task-Specialized Sentence Encoders

• URL: http://arxiv.org/abs/2312.11020v1
• Date: Mon, 18 Dec 2023 08:45:39 GMT
• Title: Information Type Classification with Contrastive Task-Specialized Sentence Encoders
• Authors: Philipp Seeberger, Tobias Bocklet, Korbinian Riedhammer
• Abstract summary: We propose the use of contrastive task-specialized sentence encoders for downstream classification. We show performance gains w.r.t. F1-score on the CrisisLex, HumAID, and TrecIS information type classification tasks.
• Score: 8.301569507291006
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: User-generated information content has become an important information source in crisis situations. However, classification models suffer from noise and event-related biases which still poses a challenging task and requires sophisticated task-adaptation. To address these challenges, we propose the use of contrastive task-specialized sentence encoders for downstream classification. We apply the task-specialization on the CrisisLex, HumAID, and TrecIS information type classification tasks and show performance gains w.r.t. F1-score. Furthermore, we analyse the cross-corpus and cross-lingual capabilities for two German event relevancy classification datasets.

Related papers

• Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective [13.56923651751788]
  We propose Causality-Guided Semantic Decoupling and Classification to mitigate the interference of task-irrelevant knowledge. We employ the Dempster-Shafer evidence theory to evaluate the uncertainty of each prediction generated by diverse semantics.
  arXiv  Detail & Related papers  (2024-10-01T09:33:45Z)
• Measuring Spurious Correlation in Classification: 'Clever Hans' in Translationese [11.478503501597967]
  We focus on topic-based spurious correlation and approach the question from two directions. We develop a measure capturing alignment of unsupervised topics with target classification labels as an indication of spurious topic information in the data. We show that our measure is the same as purity in clustering and propose a 'topic floor' (as in a 'noise floor') for classification.
  arXiv  Detail & Related papers  (2023-08-25T04:19:58Z)
• PatchMix Augmentation to Identify Causal Features in Few-shot Learning [55.64873998196191]
  Few-shot learning aims to transfer knowledge learned from base with sufficient categories labelled data to novel categories with scarce known information. We propose a novel data augmentation strategy dubbed as PatchMix that can break this spurious dependency. We show that such an augmentation mechanism, different from existing ones, is able to identify the causal features.
  arXiv  Detail & Related papers  (2022-11-29T08:41:29Z)
• Enhancing Crisis-Related Tweet Classification with Entity-Masked Language Modeling and Multi-Task Learning [0.30458514384586394]
  We propose a combination of entity-masked language modeling and hierarchical multi-label classification as a multi-task learning problem. We evaluate our method on tweets from the TREC-IS dataset and show an absolute performance gain w.r.t. F1-score of up to 10% for actionable information types.
  arXiv  Detail & Related papers  (2022-11-21T13:54:10Z)
• Association Graph Learning for Multi-Task Classification with Category Shifts [68.58829338426712]
  We focus on multi-task classification, where related classification tasks share the same label space and are learned simultaneously. We learn an association graph to transfer knowledge among tasks for missing classes. Our method consistently performs better than representative baselines.
  arXiv  Detail & Related papers  (2022-10-10T12:37:41Z)
• Distant finetuning with discourse relations for stance classification [55.131676584455306]
  We propose a new method to extract data with silver labels from raw text to finetune a model for stance classification. We also propose a 3-stage training framework where the noisy level in the data used for finetuning decreases over different stages. Our approach ranks 1st among 26 competing teams in the stance classification track of the NLPCC 2021 shared task Argumentative Text Understanding for AI Debater.
  arXiv  Detail & Related papers  (2022-04-27T04:24:35Z)
• On Guiding Visual Attention with Language Specification [76.08326100891571]
  We use high-level language specification as advice for constraining the classification evidence to task-relevant features, instead of distractors. We show that supervising spatial attention in this way improves performance on classification tasks with biased and noisy data.
  arXiv  Detail & Related papers  (2022-02-17T22:40:19Z)
• Competency Problems: On Finding and Removing Artifacts in Language Data [50.09608320112584]
  We argue that for complex language understanding tasks, all simple feature correlations are spurious. We theoretically analyze the difficulty of creating data for competency problems when human bias is taken into account.
  arXiv  Detail & Related papers  (2021-04-17T21:34:10Z)
• Predicting Themes within Complex Unstructured Texts: A Case Study on Safeguarding Reports [66.39150945184683]
  We focus on the problem of automatically identifying the main themes in a safeguarding report using supervised classification approaches. Our results show the potential of deep learning models to simulate subject-expert behaviour even for complex tasks with limited labelled data.
  arXiv  Detail & Related papers  (2020-10-27T19:48:23Z)
• On Cross-Dataset Generalization in Automatic Detection of Online Abuse [7.163723138100273]
  We show that the benign examples in the Wikipedia Detox dataset are biased towards platform-specific topics. We identify these examples using unsupervised topic modeling and manual inspection of topics' keywords. For a robust dataset design, we suggest applying inexpensive unsupervised methods to inspect the collected data and downsize the non-generalizable content.
  arXiv  Detail & Related papers  (2020-10-14T21:47:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.