CS-NET at SemEval-2020 Task 4: Siamese BERT for ComVE

• URL: http://arxiv.org/abs/2007.10830v1
• Date: Tue, 21 Jul 2020 14:08:02 GMT
• Title: CS-NET at SemEval-2020 Task 4: Siamese BERT for ComVE
• Authors: Soumya Ranjan Dash, Sandeep Routray, Prateek Varshney, Ashutosh Modi
• Abstract summary: This paper describes a system for distinguishing between statements that confirm to common sense and those that do not. We use a parallel instance of transformers, which is responsible for a boost in the performance. We achieved an accuracy of 94.8% in subtask A and 89% in subtask B on the test set.
• Score: 2.0491741153610334
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we describe our system for Task 4 of SemEval 2020, which involves differentiating between natural language statements that confirm to common sense and those that do not. The organizers propose three subtasks - first, selecting between two sentences, the one which is against common sense. Second, identifying the most crucial reason why a statement does not make sense. Third, generating novel reasons for explaining the against common sense statement. Out of the three subtasks, this paper reports the system description of subtask A and subtask B. This paper proposes a model based on transformer neural network architecture for addressing the subtasks. The novelty in work lies in the architecture design, which handles the logical implication of contradicting statements and simultaneous information extraction from both sentences. We use a parallel instance of transformers, which is responsible for a boost in the performance. We achieved an accuracy of 94.8% in subtask A and 89% in subtask B on the test set.

Related papers

• SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection [68.858931667807]
  Subtask A is a binary classification task determining whether a text is written by a human or generated by a machine. Subtask B is to detect the exact source of a text, discerning whether it is written by a human or generated by a specific LLM. Subtask C aims to identify the changing point within a text, at which the authorship transitions from human to machine.
  arXiv  Detail & Related papers  (2024-04-22T13:56:07Z)
• Findings of the WMT 2022 Shared Task on Translation Suggestion [63.457874930232926]
  We report the result of the first edition of the WMT shared task on Translation Suggestion. The task aims to provide alternatives for specific words or phrases given the entire documents generated by machine translation (MT) It consists two sub-tasks, namely, the naive translation suggestion and translation suggestion with hints.
  arXiv  Detail & Related papers  (2022-11-30T03:48:36Z)
• SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning [47.49596196559958]
  This paper introduces the SemEval-2021 shared task 4: Reading of Abstract Meaning (ReCAM) Given a passage and the corresponding question, a participating system is expected to choose the correct answer from five candidates of abstract concepts. Subtask 1 aims to evaluate how well a system can model concepts that cannot be directly perceived in the physical world. Subtask 2 focuses on models' ability in comprehending nonspecific concepts located high in a hypernym hierarchy. Subtask 3 aims to provide some insights into models' generalizability over the two types of abstractness.
  arXiv  Detail & Related papers  (2021-05-31T11:04:17Z)
• ISCAS at SemEval-2020 Task 5: Pre-trained Transformers for Counterfactual Statement Modeling [48.3669727720486]
  ISCAS participated in two subtasks of SemEval 2020 Task 5: detecting counterfactual statements and detecting antecedent and consequence. This paper describes our system which is based on pre-trained transformers.
  arXiv  Detail & Related papers  (2020-09-17T09:28:07Z)
• QiaoNing at SemEval-2020 Task 4: Commonsense Validation and Explanation system based on ensemble of language model [2.728575246952532]
  In this paper, we present language model system submitted to SemEval-2020 Task 4 competition: "Commonsense Validation and Explanation" We implemented with transfer learning using pretrained language models (BERT, XLNet, RoBERTa, and ALBERT) and fine-tune them on this task. The ensembled model better solves this problem, making the model's accuracy reached 95.9% on subtask A, which just worse than human's by only 3% accuracy.
  arXiv  Detail & Related papers  (2020-09-06T05:12:50Z)
• OCEAN: Online Task Inference for Compositional Tasks with Context Adaptation [150.1979017130774]
  We propose a variational inference framework to perform online task inference for compositional tasks. Our framework supports flexible latent distributions based on prior knowledge of the task structure and can be trained in an unsupervised manner.
  arXiv  Detail & Related papers  (2020-08-17T04:50:34Z)
• LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model [5.428461405329692]
  This paper describes our submission to subtask a and b of SemEval-2020 Task 4. For subtask a, we use a ALBERT based model with improved input form to pick out the common sense statement from two statement candidates. For subtask b, we use a multiple choice model enhanced by hint sentence mechanism to select the reason from given options about why a statement is against common sense.
  arXiv  Detail & Related papers  (2020-07-06T05:51:10Z)
• SemEval-2020 Task 4: Commonsense Validation and Explanation [24.389998904122244]
  SemEval-2020 Task 4, Commonsense Validation and Explanation (ComVE), includes three subtasks. We aim to evaluate whether a system can distinguish a natural language statement that makes sense to humans from one that does not. For Subtask A and Subtask B, the performances of top-ranked systems are close to that of humans.
  arXiv  Detail & Related papers  (2020-07-01T04:41:05Z)
• CUHK at SemEval-2020 Task 4: CommonSense Explanation, Reasoning and Prediction with Multi-task Learning [22.534520584497503]
  This paper describes our system submitted to task 4 of SemEval 2020: Commonsense Validation and Explanation (ComVE) The task is to directly validate the given sentence whether or not it makes sense and require the model to explain it. Based on BERTarchitecture with a multi-task setting, we propose an effective and interpretable "Explain, Reason and Predict" (ERP) system.
  arXiv  Detail & Related papers  (2020-06-12T13:51:12Z)
• Context-based Transformer Models for Answer Sentence Selection [109.96739477808134]
  In this paper, we analyze the role of the contextual information in the sentence selection task. We propose a Transformer based architecture that leverages two types of contexts, local and global. The results show that the combination of local and global contexts in a Transformer model significantly improves the accuracy in Answer Sentence Selection.
  arXiv  Detail & Related papers  (2020-06-01T21:52:19Z)
• CS-NLP team at SemEval-2020 Task 4: Evaluation of State-of-the-art NLP Deep Learning Architectures on Commonsense Reasoning Task [3.058685580689605]
  We describe our attempt at SemEval-2020 Task 4 competition: Commonsense Validation and Explanation (ComVE) challenge. Our system uses prepared labeled textual datasets that were manually curated for three different natural language inference subtasks. For the second subtask, which is to select the reason why a statement does not make sense, we stand within the first six teams (93.7%) among 27 participants with very competitive results.
  arXiv  Detail & Related papers  (2020-05-17T13:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.