mCPT at SemEval-2023 Task 3: Multilingual Label-Aware Contrastive Pre-Training of Transformers for Few- and Zero-shot Framing Detection

• URL: http://arxiv.org/abs/2303.09901v3
• Date: Tue, 1 Aug 2023 15:16:52 GMT
• Title: mCPT at SemEval-2023 Task 3: Multilingual Label-Aware Contrastive Pre-Training of Transformers for Few- and Zero-shot Framing Detection
• Authors: Markus Reiter-Haas, Alexander Ertl, Kevin Innerebner, Elisabeth Lex
• Abstract summary: This paper presents the winning system for the zero-shot Spanish framing detection task. Our solution employs a pre-training procedure based on multilingual Transformers. In addition to describing the system, we perform an embedding space analysis and ablation study to demonstrate how our pre-training procedure supports framing detection.
• Score: 63.540146992962526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents the winning system for the zero-shot Spanish framing detection task, which also achieves competitive places in eight additional languages. The challenge of the framing detection task lies in identifying a set of 14 frames when only a few or zero samples are available, i.e., a multilingual multi-label few- or zero-shot setting. Our developed solution employs a pre-training procedure based on multilingual Transformers using a label-aware contrastive loss function. In addition to describing the system, we perform an embedding space analysis and ablation study to demonstrate how our pre-training procedure supports framing detection to advance computational framing analysis.

Related papers

• AISPACE at SemEval-2024 task 8: A Class-balanced Soft-voting System for Detecting Multi-generator Machine-generated Text [0.0]
  SemEval-2024 Task 8 provides a challenge to detect human-written and machine-generated text. This paper proposes a system that mainly deals with Subtask B. It aims to detect if given full text is written by human or is generated by a specific Large Language Model (LLM), which is actually a multi-class text classification task.
  arXiv  Detail & Related papers  (2024-04-01T06:25:47Z)
• DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM [81.75988648572347]
  We present DetToolChain, a novel prompting paradigm to unleash the zero-shot object detection ability of multimodal large language models (MLLMs) Our approach consists of a detection prompting toolkit inspired by high-precision detection priors and a new Chain-of-Thought to implement these prompts. We show that GPT-4V with our DetToolChain improves state-of-the-art object detectors by +21.5% AP50 on MS Novel class set for open-vocabulary detection.
  arXiv  Detail & Related papers  (2024-03-19T06:54:33Z)
• KInIT at SemEval-2024 Task 8: Fine-tuned LLMs for Multilingual Machine-Generated Text Detection [0.0]
  SemEval-2024 Task 8 is focused on multigenerator, multidomain, and multilingual black-box machine-generated text detection. Our submitted method achieved competitive results, ranking at the fourth place, just under 1 percentage point behind the winner.
  arXiv  Detail & Related papers  (2024-02-21T10:09:56Z)
• Label-Efficient Object Detection via Region Proposal Network Pre-Training [58.50615557874024]
  We propose a simple pretext task that provides an effective pre-training for the region proposal network (RPN) In comparison with multi-stage detectors without RPN pre-training, our approach is able to consistently improve downstream task performance.
  arXiv  Detail & Related papers  (2022-11-16T16:28:18Z)
• Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation [80.16548523140025]
  We extend the vanilla pretrain-finetune pipeline with extra code-switching restore task to bridge the gap between the pretrain and finetune stages. Our approach could narrow the cross-lingual sentence representation distance and improve low-frequency word translation with trivial computational cost.
  arXiv  Detail & Related papers  (2022-04-16T16:08:38Z)
• Zero-Shot Cross-lingual Semantic Parsing [56.95036511882921]
  We study cross-lingual semantic parsing as a zero-shot problem without parallel data for 7 test languages. We propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-Logical form paired data. Our system frames zero-shot parsing as a latent-space alignment problem and finds that pre-trained models can be improved to generate logical forms with minimal cross-lingual transfer penalty.
  arXiv  Detail & Related papers  (2021-04-15T16:08:43Z)
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances [77.28192419848901]
  We propose a simple, yet effective method named contrasting shifted instances (CSI) In addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Our experiments demonstrate the superiority of our method under various novelty detection scenarios.
  arXiv  Detail & Related papers  (2020-07-16T08:32:56Z)
• Context-Transformer: Tackling Object Confusion for Few-Shot Detection [0.0]
  We propose a novel Context-Transformer within a concise deep transfer framework. Context-Transformer can effectively leverage source-domain object knowledge as guidance. It can adaptively integrate these relational clues to enhance the discriminative power of detector.
  arXiv  Detail & Related papers  (2020-03-16T16:17:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.